Stephane Durfort
03/22/2023, 4:22 PMOmegaConfigLoader
to eventually replace the TemplateConfigLoader
in my pipeline, I noticed that
β’ variable interpolation does not seem to be applied on nested parameters (as in the model_options
example mentioned in the documentation)
β’ using kedro run --params
only update parameters but does not propagate to references of these parameters in the configuration ?
Am I doing something wrong ?# conf/base/parameters_globals.yml
global: test_1
data:
size: 0.2
# conf/base/parameters.yml
param: ${global}
nested:
param: ${global}
model_options:
test_size: ${data.size}
random_state: 3
# conf/base/parameters/demo.yml
demo: ${global}
demo_nested:
param: ${global}
# src/test_omegaconf/pipelines/demo/pipeline.py
def create_pipeline(**kwargs) -> Pipeline:
return pipeline(
[
node(
func=demo_node,
inputs=["parameters", "params:demo_nested"],
outputs=None,
name="demo"
)
]
)
# src/test_omegaconf/pipelines/demo/nodes.py
def demo_node(parameters, nested_parameters):
print(type(parameters))
pprint(parameters)
print(type(nested_parameters))
pprint(nested_parameters)
β― kedro run
INFO Running node: demo: demo_node([parameters,params:demo_nested]) -> None
<class 'dict'>
{'data': {'size': 0.2},
'demo': 'test_1',
'demo_nested': {'param': '${global}'},
'global': 'test_1',
'model_options': {'test_size': '${data.size}', 'random_state': 3},
'nested': {'param': '${global}'},
'param': 'test_1'}
<class 'omegaconf.dictconfig.DictConfig'>
{'param': '${global}'}
β― kedro run --params global:test_2
INFO Running node: demo: demo_node([parameters,params:demo_nested]) -> None
<class 'dict'>
{'data': {'size': 0.2},
'demo': 'test_1',
'demo_nested': {'param': '${global}'},
'global': 'test_2',
'model_options': {'test_size': '${data.size}', 'random_state': 3},
'nested': {'param': '${global}'},
'param': 'test_1'}
<class 'omegaconf.dictconfig.DictConfig'>
{'param': '${global}'}
0.18.6
Nok Lam Chan
03/22/2023, 4:40 PMStephane Durfort
03/22/2023, 4:53 PMprint(nested_parameters["param"])
-> test_1
parameters
returns a dict while params:xxx
returns an omegaconf object ?Nok Lam Chan
03/22/2023, 5:17 PM${global}
reference.
For 1. if you access parameters["demo_nested"]["param"]
it will still be the correct value.
2 seems to be a bug.Stephane Durfort
03/22/2023, 5:21 PMNok Lam Chan
03/22/2023, 5:38 PMOmegaConfigLoader
, any feedback is very welcomed.Stephane Durfort
03/22/2023, 5:49 PMOmegaConfigLoader
when it will allow to configure catalogs as well (same concern as https://kedro-org.slack.com/archives/C03RKP2LW64/p1679009485518709) and I realize now that 1/ is not an issue but my misinterpretation of the display of parameters versus their use.Nok Lam Chan
03/22/2023, 5:58 PM