Hello kedro team, I'm actually working on project ...
# questions
y
Hello kedro team, I'm actually working on project of Churn prediction and i finish all pipeline and the job work fine. I work on Kedro 0.16.5 because it was compatible with some packages on our environment. After packging the project now i'm able to run it from command line with:
Copy code
python3 -m project_name.run
But i have question about parameters. When i run the packaged project i can't anymore pass parameters to the project or modifiy the parameters.yml so my question is how to pass arguments when i run a packaged kedro project ?
d
so this is much nicer in forthcoming 0.19.x release but you would have to inject them into the packaged wheel somehow on this version
n
Ok I found out I accidentally muted the channel notification😂 I like this question! @datajoely is correct that in 0.19.x it would be even easier to dynamically change the config location. But you can still work with changing parameters in older versions. I cannot guarantee it works
0.16.5
since this is really old, more than 2 years. It may be surprising but
python -m module
actually accepts the same arguments as
kedro run
, this isn’t documented clearly. TL;DR; So if you can do
kedro run --params=key:value
, you can do the same with
python -m custom_pacakge --params=key:value
to override runtime parameters. The gotcha of the current package project is that. During development, your
conf/base/parameters.yml
is inside the root directory, however, when you move to package mode the parameters have to move into
src/conf/base/parameters.yml
or you have to do some tricks in
settings.py
to read config from somewhere else.
y
So i tried the
python -m custom_pacakge --params=key:value
but it didn't work for me. And for the file
parameters.yml
i set it in
conf/base/parameters.yml
but in another machine different from the one used before packaging.
n
Sorry, I don’t have a quick solution in
0.16.x
since the codebase is quite different. I test it with
0.18.3
and I expect this will work for all
0.18.x
series or maybe even older versions.
😔 1
y
Finally, i was able to use parameters but by passing them as environment variables and use the TemplatedConfigLoader in the Hook
Copy code
def register_config_loader(self, conf_paths: Iterable[str]) -> ConfigLoader:
    return TemplatedConfigLoader(conf_paths,
                                 globals_pattern="*globals.yml",
                                 globals_dict={
                                     "date_process": datetime.now().strftime("%Y%m%d%H%M%S"),
                                     "MODEL_OUTPUT": os.getenv("MODEL_OUTPUT"),
                                     "PROJECT_DIR": os.getenv("PROJECT_HDFS_DIR"),

                                 })
and by using the global.yml file i can change some inputs depending if i'm in dev or prod env