https://kedro.org/ logo
#questions
Title
# questions
l

Lucas Hattori

06/22/2023, 3:10 PM
Hi team, I’m starting to use
kedro-mlflow
for the first time in a project. Would this be also a appropriate theme for questions here? 😅 if so, regarding parameters, my kedro project has a lot of parameters. Many of them are not crucial for me to be logged in mlflow experiments. How can I easily select which parameters I’d like to have it logged? I have an idea on how to it if I were to build the mlflow hooks from scratch, but I’d love to leverage
kedro-mlflow
for simplicity
n

Nok Lam Chan

06/22/2023, 3:17 PM
kedro-mlflow
is created by one of our contributor @Yolan Honoré-Rougé and their team. I am not familiar with the plugin myself, let’s see if there are some advance users around know about this. : )
thankyou 1
m

marrrcin

06/22/2023, 6:52 PM
Also, there's a #plugins-integrations channel for those
👍 1
thankyou 1
y

Yolan Honoré-Rougé

06/23/2023, 8:11 PM
Hi, this is quite an uncommon use case.
kedro-mlflow
is quite opinionated and forces you to log every "parameters" declared in your pipeline to ensure reproducibility (which is one goal of mlflow :)). If you really want to "trick" kedro-mlflow, you can declare these parameters in a
extra_parameters.yml
, and then load this file in the catalog trhough a
YamlDataSet
but I would not recommend it. I think the best way would be simply to log everything, and to filter the parameters you don't want to display directly in the mlflow ui.
👍 1
l

Lucas Hattori

06/23/2023, 8:15 PM
Thanks @Yolan Honoré-Rougé ! Yes, indeed ideal would be to declare all parameters to ensure reproducibility, and it makes sense for `kedro-mlflow`to force you to do that! I think this gives me an idea that maybe I should be smarter when selecting which parameters I really need to use in each node (instead of, for instance, logging the whole
parameters
dictionary, to log just a subset of it).
y

Yolan Honoré-Rougé

06/23/2023, 8:48 PM
Ah sure. I think passing all
parameters
in a node is an antipattern and this is likely inherited from old kedro versions. My intuition is that it should not be encouraged (nor even permitted?) in kedro.
2 Views