Hi team, I’m starting to use

kedro-mlflow

for the first time in a project. Would this be also a appropriate theme for questions here? 😅 if so, regarding parameters, my kedro project has a lot of parameters. Many of them are not crucial for me to be logged in mlflow experiments. How can I easily select which parameters I’d like to have it logged? I have an idea on how to it if I were to build the mlflow hooks from scratch, but I’d love to leverage

kedro-mlflow

for simplicity

kedro-mlflow

is created by one of our contributor @Yolan Honoré-Rougé and their team. I am not familiar with the plugin myself, let’s see if there are some advance users around know about this. : )

Also, there's a #plugins-integrations channel for those

Hi, this is quite an uncommon use case.

kedro-mlflow

is quite opinionated and forces you to log every "parameters" declared in your pipeline to ensure reproducibility (which is one goal of mlflow :)). If you really want to "trick" kedro-mlflow, you can declare these parameters in a

extra_parameters.yml

, and then load this file in the catalog trhough a

YamlDataSet

but I would not recommend it. I think the best way would be simply to log everything, and to filter the parameters you don't want to display directly in the mlflow ui.

Thanks @Yolan Honoré-Rougé ! Yes, indeed ideal would be to declare all parameters to ensure reproducibility, and it makes sense for `kedro-mlflow`to force you to do that! I think this gives me an idea that maybe I should be smarter when selecting which parameters I really need to use in each node (instead of, for instance, logging the whole

parameters

dictionary, to log just a subset of it).

Ah sure. I think passing all

parameters

in a node is an antipattern and this is likely inherited from old kedro versions. My intuition is that it should not be encouraged (nor even permitted?) in kedro.