What is the cleanest way to run the entire pipeline multiple times? I have a parameter,

observed_date = '2024-10-01'

, defined in

parameters.yml

, that I use to run the pipeline. At the end of the pipeline, the output is saved to or replaced in a SQL table. Now, I want to loop over this pipeline for every 5 day from Jan 2022 till October 2024. Manually, this would require updating the

parameters.yml

file each time I want to change the date and rerun the pipeline (

kedro run

). I don't want to introduce a loop directly into the pipeline, as it’s cleaner when

observed_date

is treated as a single date rather than a list of dates. However, I’d like to find a clean way to loop over different dates, running

kedro run

for each date.

Someone will reply to you shortly. In the meantime, this might help:

You could loop in a script and pass the observed_date as a parameter: kedro run --params observed date=observed date

@Paul Weiss Thank you! I also just read about the parameters at run time capability. I should be able to perform a loop for the observed date I want. https://docs.kedro.org/en/stable/configuration/parameters.html