Felipe Vianna
07/06/2023, 1:29 PMmy_param: ${my_param_in_globals.yml}
for some reason we can parse params like that just from base globals.yml. and when i try to get from live/globals.yml it’s not taken.Juan Luis
07/06/2023, 1:34 PMConfigLoader
or something else? also, what version of Kedro are you using?Felipe Vianna
07/06/2023, 1:36 PMfrom kedro.config import TemplatedConfigLoader
import os
CONFIG_LOADER_CLASS = TemplatedConfigLoader
CONFIG_LOADER_ARGS = {"globals_pattern": "*globals.yml",
"globals_dict": os.environ,
}
Juan Luis
07/06/2023, 1:51 PMglobals_pattern
🤔 what about
"globals_pattern": "**/globals.yml",
@Felipe Vianna?Felipe Vianna
07/06/2023, 1:57 PMMerel
07/06/2023, 3:20 PMbase
the values aren’t passed on kedro run
?Felipe Vianna
07/06/2023, 3:57 PMkedro run --env live
Merel
07/06/2023, 4:14 PMbefore_pipeline_run
hook which has run_params
and contains env
Felipe Vianna
07/06/2023, 4:20 PMkedro run --env ENV
Merel
07/06/2023, 4:20 PMclass PipelineHooks:
@property
def _logger(self):
return logging.getLogger(self.__class__.__name__)
@hook_impl
def before_pipeline_run(
self, run_params: dict[str, Any], pipeline: Pipeline, catalog: DataCatalog
) -> None:
self._logger.warning(f"ENV: {run_params['env']}")
Felipe Vianna
07/06/2023, 4:22 PMKEDRO_ENV=test kedro run --pipeline live_pipe --params=run_end_date:"2023-07-02 07:30:00"
the expected result is that the run_end_date
would be taken from the CLI params instead of the globals.ymlMerel
07/07/2023, 8:16 AMFelipe Vianna
07/07/2023, 8:51 AMFailed to format pattern '${run_end_date}': no config value found, no default provided
Juan Luis
07/07/2023, 11:08 AMglobals
not being picked up? happy to have another look at this and try to craft a reproducerFelipe Vianna
07/07/2023, 11:23 AMKEDRO_ENV=live kedro run ….
and then in the py file I used the config loader passing the env I could get with os.getenv(). that forced the usage of the right globals.Juan Luis
07/07/2023, 11:29 AMFelipe Vianna
07/07/2023, 11:38 AM