Cory Maklin
08/28/2023, 4:57 PMparams in the catalog..yml in order to make the paths dynamic?Erwin
08/28/2023, 5:40 PMEmilio Gagliardi
08/28/2023, 6:09 PMCory Maklin
08/28/2023, 6:29 PMdatajoely
08/28/2023, 6:43 PMkedro-mlflow plug-in?datajoely
08/28/2023, 6:43 PMLodewic van Twillert
08/29/2023, 7:27 AMkedro-mlflow plugin๐Lodewic van Twillert
08/29/2023, 7:38 AMparams to make paths dynamic, using namespaces: few days ago a similar question was asked and in the reply thread we might have a possible implementation for you
https://kedro-org.slack.com/archives/C03RKP2LW64/p1692872422344789
Without any additional work, no you cannot refer to params in the catalog.yml at the moment.
-- However, assuming you are indeed using the kedro-mlflow plugin and logging mlflow artifacts using the <http://kedro_mlflow.io|kedro_mlflow.io>.artifacts.MlflowArtifactDataSet - then I wouldn't think you need to refer to your params.yml to make the filepath dynamic? Since the dataset would be logged as an artifacts to the mlflow run with the params you need? ๐ค
If you share some more about how you are setting up your concurrent runs, I suspect you could get away with just using namespaces by following the example in the docs here: https://docs.kedro.org/en/stable/data/data_catalog.html#example-3-generalise-datasets-using-namespaces-into-one-dataset-factorydatajoely
08/29/2023, 8:52 AM