Hey team! I hope you are all doing well! 🙂 I am looking for a possibility of an incremental dataset. For example I do not want to re-create all partitions, but only the partitions which were not present before. Does kedro have something inbuild to cope with such incremental savings?

hi @Paul Mora, yes, Kedro supports partitioned and incremental datasets. You can find more information here: https://github.com/kedro-org/kedro-plugins/tree/main/kedro-datasets/kedro_datasets/partitions https://docs.kedro.org/projects/kedro-datasets/en/kedro-datasets-3.0.1/api/kedro_datasets.partitions.IncrementalDataset.html

Ah, that sounds like something I could have found myself - thanks a ton!

@Paul Mora speaking of which, was it that you tried to look it up in our docs (or Google, or ChatGPT) and you didn't find it? or you came here to Slack directly?

Tried to look it up quickly, but didnt find it intiially. I should have better looked!

no worries! we proud ourselves on giving quick support in this Slack 😄 just asking because we have a number of SEO issues in our docs