is it possible to override globals in omegaconfig ...
# questionse
Erwin
09/01/2023, 2:17 PMis it possible to override globals in omegaconfig loader with extra params? [Im testing kedro 0.18.13]
đ 1
m
marrrcin
09/01/2023, 2:31 PMmarrrcin
09/01/2023, 2:32 PM[EDITED: I havenât noticed the âextra paramsâ]
Globals in OmegaConfigLoader are available from 0.18.13, but Iâm not sure whether they play along the extra params
https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-use-global-variables-with-the-omegaconfigloader
j
Juan Luis
09/01/2023, 2:33 PMthe good thing of living in the bleeding edge đ
đ¤ 2
m
marrrcin
09/01/2023, 2:34 PMWhat is your use case @Erwin?
e
Erwin
09/01/2023, 2:43 PMHi!
I have in
conf/base/parameters/balls.yml Copy code
ball_level:
start_date: ${globals:start_date} #kedro 0.18.13
#start_date: ${start_date} #kedro 0.18.12conf/base/globals.ymlconf/base/parameters_globals.yml Copy code
start_date: 202307 Copy code
from kedro.framework.session import KedroSession
from kedro.framework.startup import bootstrap_project
from kedro.runner import ThreadRunner
bootstrap_project(project_root)
params = {"start_date": previous_month}
with KedroSession.create(
project_path=project_root,
env=env,
extra_params=params,
) as session:
session.run(pipeline_name=pipeline, runner=ThreadRunner())m
marrrcin
09/01/2023, 2:45 PMPlease show
settings.pye
Erwin
09/01/2023, 2:47 PM Copy code
from kedro.config import OmegaConfigLoader # noqa
CONFIG_LOADER_CLASS = OmegaConfigLoader
# Keyword arguments to pass to the `CONFIG_LOADER_CLASS` constructor.
CONFIG_LOADER_ARGS = {
"config_patterns": {
"spark": ["spark*/"],
"parameters": ["parameters*", "parameters*/**", "**/parameters*"],
}
}m
marrrcin
09/01/2023, 2:50 PMSo in 0.18.12 it was not using globals, it was actually a side-effect of variable interpolation. Maybe @Ankita Katiyar can chip in here đ¤
a
Ankita Katiyar
09/01/2023, 2:52 PMYep, you can override
parametersglobalsglobals.ymlm
marrrcin
09/01/2023, 2:53 PMBut do the
extra_paramsa
Ankita Katiyar
09/01/2023, 2:53 PMYeah, they should work for regular variable interpolation. Only for parameters though I think
m
marrrcin
09/01/2023, 2:54 PMI guess that should answer the question @Erwin
a
Ankita Katiyar
09/01/2023, 2:55 PMWhat Erwin was doing previously would still work with
0.18.13globals.ymlâď¸ 2
e
Erwin
09/01/2023, 2:59 PMI see so, this works:
Copy code
params = {"ball_level.start_date": previous_month}a
Ankita Katiyar
09/01/2023, 3:17 PMYeah, correct.
7 Views