https://kedro.org/ logo
#plugins-integrations
Title
# plugins-integrations
m

Marc Gris

08/31/2023, 3:05 PM
Hi everyone, I’ve started to use
kedro-mlflow
and am under the impression that there is some conflict / incompatibility with the dataset factory. Having moved from
Copy code
"models.{model}.embeddings":
    type: pickle.PickleDataSet
    filepath: "data/07_model_output/{model}_embeddings.pkl"
    backend: joblib
    versioned: True
    layer: model_output
to
Copy code
"models.{model}.embeddings":
  type: kedro_mlflow.io.artifacts.MlflowArtifactDataSet
  data_set:
    type: pickle.PickleDataSet
    filepath: "data/07_model_output/{model}_embeddings.pkl"
    backend: joblib
    versioned: True
  layer: model_output
results in artifacts being saved without the model named properly interpolated, i.e “litteraly” saved as
{model}_embeddings.pkl
Any comment ? Anything I’m missing ? Many thanks in advance, Regards, M.
j

Juan Luis

08/31/2023, 3:07 PM
hi @Marc Gris! just to make sure, are you using the
OmegaConfigLoader
and
the latest version of Kedro?
m

Merel

08/31/2023, 3:08 PM
It should work with any config loader
a

Ankita Katiyar

08/31/2023, 3:08 PM
I see, this is a bug on our end
m

Merel

08/31/2023, 3:08 PM
dataset factories isn’t specific to OCL
this 1
a

Ankita Katiyar

08/31/2023, 3:09 PM
When we resolve the config it goes through each entry and fills out the placeholder, but here the filepath is nested
Creating a ticket for this! Thanks for flagging this @Marc Gris
👍🏼 1
👍 1
m

Marc Gris

08/31/2023, 3:20 PM
Hi everyone Thanks for your swift reactions / replies. I’ve tried both with 0.18.12 & 0.18.13 and get the same issue. Just to be extra clear: The dataset factory patterns works fine, until I use
<http://kedro_mlflow.io|kedro_mlflow.io>.artifacts.MlflowArtifactDataSet
I hope this is a trivial bug to fix 🙂 @Ankita Katiyar Could you please share the ticket here, so that I may “follow” it and be automatically informed when it is fixed ? Many thanks in advance, Regards M.
👍 2
a

Ankita Katiyar

08/31/2023, 3:33 PM
Here’s the issue https://github.com/kedro-org/kedro/issues/2992 🙂 The fix should be simple will open a PR for it
m

Marc Gris

08/31/2023, 3:34 PM
Thx @Ankita Katiyar 🙂 🙏🏼
Hi @Ankita Katiyar I hope you’re well 🙂 Do you have any “guesstimate” as to when your changes will be accepted & merged into main ? Thx M
a

Ankita Katiyar

09/06/2023, 10:52 AM
Hey @Marc Gris, we’re dealing with CI issues at the moment 😅 but it’s ready to go!
m

Marc Gris

09/06/2023, 2:08 PM
Great ! Thx @Ankita Katiyar 🙂
a

Ankita Katiyar

09/07/2023, 4:56 PM
@Marc Gris it’s merged!
m

Marc Gris

09/07/2023, 5:08 PM
🥳 Thx @Ankita Katiyar
2 Views