Hi everyone! I am new to kedro, and I like the idea of standardizing the structure of data science projects a lot. However, sometimes I am still a little bit confused what the intended way of implementation in Kedro looks like. For example, let's say I want to do a standard evaluation of n models on m datasets - is this conceptually a single pipeline, that runs the whole analysis for all nxm combinations? Or is it better implemented by dynamically creating one pipeline per model/dataset/model+data combination? Or is this even a use case for different conv environments? Any tips and suggestions are greatly appreciated 🙂

i guess it depends on how different the dataset is but if all the data is the same schema (e.g. maybe just subsets of a larger dataset) then you could do as you say one pipeline with different instantiations My favorite way to do this is namespacing: https://docs.kedro.org/en/stable/build/namespaces/ And on top of that, dataset factories: https://docs.kedro.org/en/stable/catalog-data/kedro_dataset_factories/ So your "base" pipeline will take as inputs things like

data

and output a

model

and maybe some

metrics

Then when you namespace for your data subset

alpha

, your namespaced pipeline will look for

alpha.data

and output

alpha.model

and

alpha.metrics

You can then use the dataset factory to store all of these in the same way without blowing up your catalog The last piece of the puzzle is defining a big list of all the namespaces - this is kind of up to you. I usually do it in

settings.py

so I can reuse it throughout the project

I recently faced a similar problem where I had to evaluate 2 architectures (XGBoost vs Neural Net) on train/val splits and test data which was another external dataset. For training and eval, I had one pipeline for dataset A, then I had a sub-pipeline returning train and validation splits, then the common splits were fed to 2 separate pipelines for the 2 architectures, which were again split into training and inference sub-pipes When I evaluated on the test dataset (which had the same input features, but different preprocessing to get to the features) I designed a specific pipeline and then re-used the previous inference sub-pipes. Namespaces are a good way to develop composable sub-pipelines within one pipeline.py, but be sure to understand the

prefix_datasets_with_namespace

flag (https://docs.kedro.org/en/stable/build/namespaces/#what-is-a-namespace) to avoid confusion with the parameters names. At the end in the pipeline registry I had 4 names: • train_modelA: preprocess datasetA, split, train model A (3 sub-pipes) • train_modelB: preprocess datasetA, split, train model B (3 sub-pipes) • test_modelA: preprocess datasetB, eval model A on test (2 sub-pipes) • test_modelB: preprocess datasetB, eval model B on test (2 sub-pipes) Additionally, I could've wired end-to-end pipelines, where a model was both trained and tested. But I decided to leave test dataset analyses separate for bias control. For more complex evaluations and more similar models, I would consider a different wiring that would mirror better what the goal of my experiments (e.g. finding good hyperparameters) rather than evaluating a model.

Thanks, that was very helpful :)