Yousri
11/28/2022, 3:27 PMpython3 -m project_name.run
But i have question about parameters. When i run the packaged project i can't anymore pass parameters to the project or modifiy the parameters.yml so my question is how to pass arguments when i run a packaged kedro project ?datajoely
11/28/2022, 4:10 PMNok Lam Chan
11/29/2022, 3:49 PM0.16.5
since this is really old, more than 2 years. It may be surprising but python -m module
actually accepts the same arguments as kedro run
, this isn’t documented clearly.
TL;DR; So if you can do kedro run --params=key:value
, you can do the same with python -m custom_pacakge --params=key:value
to override runtime parameters.
The gotcha of the current package project is that.
During development, your conf/base/parameters.yml
is inside the root directory, however, when you move to package mode the parameters have to move into src/conf/base/parameters.yml
or you have to do some tricks in settings.py
to read config from somewhere else.Yousri
11/29/2022, 4:05 PMpython -m custom_pacakge --params=key:value
but it didn't work for me.
And for the file parameters.yml
i set it in conf/base/parameters.yml
but in another machine different from the one used before packaging.Nok Lam Chan
11/29/2022, 4:09 PM0.16.x
since the codebase is quite different. I test it with 0.18.3
and I expect this will work for all 0.18.x
series or maybe even older versions.Yousri
12/07/2022, 10:48 AMdef register_config_loader(self, conf_paths: Iterable[str]) -> ConfigLoader:
return TemplatedConfigLoader(conf_paths,
globals_pattern="*globals.yml",
globals_dict={
"date_process": datetime.now().strftime("%Y%m%d%H%M%S"),
"MODEL_OUTPUT": os.getenv("MODEL_OUTPUT"),
"PROJECT_DIR": os.getenv("PROJECT_HDFS_DIR"),
})
and by using the global.yml file i can change some inputs depending if i'm in dev or prod env