Hi all, New to kedro. I have one issue download data incrementally. I'd like to download monthly data from an API and store the data in files. I'd like to always download all available months since the last run. How can achieve this behavior using incremental dataset tools? The alternative will be to be to use a custom file mask / checkpoint file that allows me to know which months have been downloaded. Thabks

I think your best bet would be to create a custom incremental dataset that handles the logic of checking which months have already been downloaded and fetching new data. https://docs.kedro.org/en/stable/data/how_to_create_a_custom_dataset.html

Thanks for your response