Rafał Nowak
02/24/2023, 8:20 AM--params "section1.section2.name:value"
where section1.section2
is defined in parameters.yml
so it seems that there is some tree section1.section2
with some parameters. I would like to change only one of them.
I think I know that kedro is not able to override only one paramter in the tree. I have to overrside the full root which is not user friendly in the CLI.
I see that since kedro 0.18.5, one can use OmegaConf now. Does it change this limitatiom?
If so, is it possible to use global_patterns
like it was in TemplatedConfigLoader ?catalo/*.yml
files.
One cannot define the string in parametes/*.yml
since they are not visible.
One cannot also define such string in catalog/*.yml
since kedro fails when loading such catalog item; for example some layer popup
eror appears.
I thought that _name
would be solution, but not.Vassilis Kalofolias
02/24/2023, 3:00 PMkedro run --config config.yml
, where you have defined# content of config.yml
run:
params:
section1:
section2:
name: value
• kedro.framework.session.Session.KedroSession(..., extra_params={"section1":{"section2":{name:value}}})
. This will replace all levels of the tree with the overriding parameters, while keeping the rest intact.
• you can not use add_all
or add
or add_feed_dict
easily as it doesn't replace all levels of the tree.kedro run --params "section1.section2.name:value"
extra_params
(keeping the rest of the tree intact).extra_params
it doesn't behave as the one of kedro.framework.session.Session.KedroSession
and instead it replaces the high level tree.
So to make it work you have to run the full script:
# run_experiment.py
from kedro.framework.session import KedroSession
from kedro.framework.startup import bootstrap_project
from pathlib import Path
extra_params = {
section1: {
section2: {
name: value
}
}
}
project_path = Path(".").resolve()
metadata = bootstrap_project(project_path)
session = KedroSession.create(metadata.package_name, project_path, env=None, extra_params=extra_params)
context = session.load_context()
catalog = context.catalog
assert catalog.load("params:section1.section2.name") == value
# check that other params have not disappeared
assert catalog.load("params:section1.section2.another_name") == another_value
Rafał Nowak
02/25/2023, 12:54 PMkedro run --params "section1.section2.name:value"
I think that OmegaConf would be better solution. However I still do not know how to add interpolation string into catalog/*.yml
files.
One cannot define globals in this folder since Kedro is trying to create DataSet objects.Vassilis Kalofolias
02/27/2023, 6:17 PM