Kedro is an open-sourced Python framework for creating maintainable and modular data science code.

Kedro

Hi y'all! Say I have a very standard pipeline like this: `get-data -&gt; train-model -&gt; evaluate-model`. Now, the model can be any of sklearn's models, all with the same interface. What I'd like to do is, from a list of models specified in `parameters` , run many instances of this pipeline each with one model of the list (of course, I'd like pipelines to run in parallel).

I can use modular pipelines to instantiate the pipeline many times, but I'm not sure how to use the model list in the parameters file. Any ideas?

I think this is what you need: <https://github.com/datajoely/modular-spaceflights/tree/main/src/modular_spaceflights/pipelines/modelling>

Check also the function `new_modeling_pipeline` <https://github.com/datajoely/modular-spaceflights/blob/main/src/modular_spaceflights/pipelines/modelling/pipeline.py|here> .

In this example, the list of model types is hardcoded in <https://github.com/datajoely/modular-spaceflights/blob/3117920d6cee722872ed1d76a471cdbc43d1dca1/src/modular_spaceflights/pipeline_registry.py#L24|pipeline_registry>.

If you want this to be read from the parameters, I _assume_ that you need to use a hook `after_catalog_created`  to make sure that parameters are already parsed when `pipeline_registry` runs (haven't done this though).

Thanks <@U04M6VA7Z6U>! That was helpful. I managed to read from the parameters by programmatically importing it using `ConfigLoader` .