Hi everybody, is it there something equivalent to

pandas.SQLQueryDataSet

in spark? can I get the same functionality in spark? I can not make queries with

spark.SparkJDBCDataSet

, am I missing something? Thanks in advance!

Hello - we don’t support that natively, you could easily subclass

spark.SparkJDBCDataSet

if you’d like - but we prefer DataFrame like interfaces in Kedro 🙂

hehehe I was thinking exactly that, but wanted to ask first! Thanks for the quick response!

Fundamentally the UD part of CRUD doesn’t make lots of sense in Kedro world so we prefer keeping transformation in DataFrame world

yeah, i'll think about it, I agree that's not a "kedronic" solution