Lucas Hattori06/22/2023, 3:10 PM
for the first time in a project. Would this be also a appropriate theme for questions here? 😅 if so, regarding parameters, my kedro project has a lot of parameters. Many of them are not crucial for me to be logged in mlflow experiments. How can I easily select which parameters I’d like to have it logged? I have an idea on how to it if I were to build the mlflow hooks from scratch, but I’d love to leverage
Nok Lam Chan06/22/2023, 3:17 PM
is created by one of our contributor @Yolan Honoré-Rougé and their team. I am not familiar with the plugin myself, let’s see if there are some advance users around know about this. : )
marrrcin06/22/2023, 6:52 PM
Yolan Honoré-Rougé06/23/2023, 8:11 PM
is quite opinionated and forces you to log every "parameters" declared in your pipeline to ensure reproducibility (which is one goal of mlflow :)). If you really want to "trick" kedro-mlflow, you can declare these parameters in a
, and then load this file in the catalog trhough a
but I would not recommend it. I think the best way would be simply to log everything, and to filter the parameters you don't want to display directly in the mlflow ui.
Lucas Hattori06/23/2023, 8:15 PM
dictionary, to log just a subset of it).
Yolan Honoré-Rougé06/23/2023, 8:48 PM
in a node is an antipattern and this is likely inherited from old kedro versions. My intuition is that it should not be encouraged (nor even permitted?) in kedro.