CONFIGS CONSOLIDATIONS INTERPOLATIONS in conf config yml `r Kedro #questions

CONFIGS CONSOLIDATIONS / INTERPOLATIONS in conf/...

Marc Gris

06/21/2023, 9:31 AM

CONFIGS CONSOLIDATIONS / INTERPOLATIONS in conf/config.yml random_state: 42
and in conf/model_training.yml random_state: ${random_state}

Copy code

kedro run 
>>> [...] 
TypeError: Cannot cast scalar from dtype('<U15') to dtype('int64') according to the rule 'safe'

If I get this correctly, the consolidation / interpolation process resulted in

random_state

being assigned the value

"42"

instead of

Granted, I could easily circumvent this issue with

int(params['random_state'])

, but I’m curious and would like to know if this is an expected behavior, and whether there is a more robust / elegant way of handling it. Thx in advance M

datajoely

06/21/2023, 9:31 AM

which config loader are you using ?

datajoely

06/21/2023, 9:33 AM

you can also try this

random_state: !!int 42

datajoely

06/21/2023, 9:34 AM

I think that will coerce it properly https://pyyaml.org/wiki/PyYAMLDocumentation

Marc Gris

06/21/2023, 9:41 AM

Thx @datajoely I wasn’t aware of this syntax for type coercion 🙂 Regarding the loader: I’m just using the default, plain vanilla

ConfigLoader

Could that be the issue ? Shall I use another loader ? Thx

Marc Gris

06/21/2023, 9:46 AM

Hmmm… It looks like it could indeed be the issue. I’ve just tried to do

int(params['random_state']

and got

ValueError: invalid literal for int() with base 10: '${random_state}'

I guess variable interpolation requires another config loader… 😅

Marc Gris

06/21/2023, 9:48 AM

ah… If only I read (or memorized) documentations better: https://docs.kedro.org/en/stable/configuration/advanced_configuration.html

datajoely

06/21/2023, 9:53 AM

Yes - but longer term the default ConfigLoader (which will be replaced) will have this natively

Marc Gris

06/21/2023, 10:52 AM

Oh ! great to hear 👍

Open in Slack

Previous Next