Hey, is there any way to use an IncrementalDataSet with a SQL query to incrementally fetch partitions from RDBMS tables, or it works only with object storage partitioned data?

I don’t think this is supported natively - @datajoely may have better idea. It was designed to work with files - is an example of how it can be used.

no we don’t natively support this - if you look at our DeltaLake docs we talk about how to do the

UD

part of

CRUD

in Kedro, but it’s not a fantastic fit for our load/save pattern https://docs.kedro.org/en/stable/tools_integration/pyspark.html#spark-and-delta-lake-interaction

ok thanks for the information :)