Cory Maklin
08/28/2023, 4:57 PMparams
in the catalog..yml
in order to make the paths dynamic?Erwin
08/28/2023, 5:40 PMEmilio Gagliardi
08/28/2023, 6:09 PMCory Maklin
08/28/2023, 6:29 PMdatajoely
08/28/2023, 6:43 PMkedro-mlflow
plug-in?Lodewic van Twillert
08/29/2023, 7:27 AMkedro-mlflow
plugin๐params
to make paths dynamic, using namespaces: few days ago a similar question was asked and in the reply thread we might have a possible implementation for you
https://kedro-org.slack.com/archives/C03RKP2LW64/p1692872422344789
Without any additional work, no you cannot refer to params
in the catalog.yml
at the moment.
-- However, assuming you are indeed using the kedro-mlflow
plugin and logging mlflow artifacts using the <http://kedro_mlflow.io|kedro_mlflow.io>.artifacts.MlflowArtifactDataSet
- then I wouldn't think you need to refer to your params.yml to make the filepath dynamic? Since the dataset would be logged as an artifacts to the mlflow run with the params you need? ๐ค
If you share some more about how you are setting up your concurrent runs, I suspect you could get away with just using namespaces
by following the example in the docs here: https://docs.kedro.org/en/stable/data/data_catalog.html#example-3-generalise-datasets-using-namespaces-into-one-dataset-factorydatajoely
08/29/2023, 8:52 AM