https://kedro.org/ logo
#questions
Title
# questions
s

Safouane Chergui

11/17/2022, 10:19 AM
Hello everyone, Is there a way to use parameters that are passed to
kedro run
as an input to a node ? Here is a quick example: • Kedro run command:
kedro run --pipeline my_pipeline --params first_param:first_value
• I’d like to use first_param as an input to a node without having to put it in parameters.yml just to use it as an input to my node. If not, is there a way to use it directly into code ?
Copy code
Pipeline([
            node(
                do_something,
                 inputs="first_param",
                 outputs="some_output"
            )
       ])
Thanks 👍
f

FlorianGD

11/17/2022, 10:20 AM
You can use
params:first_param
as an input and it should work without having to declare it in
parameters.yml
👍 1
s

Safouane Chergui

11/17/2022, 10:24 AM
Actually, that was the first thing that I tried and I got this error: ValueError: Pipeline input(s) {‘params:first_param’} not found in the DataCatalog
f

FlorianGD

11/17/2022, 10:29 AM
Which Kedro version are you using?
s

Safouane Chergui

11/17/2022, 10:29 AM
I’m using 0.17.6
f

FlorianGD

11/17/2022, 10:30 AM
I'd recommend switching to Kedro 0.18.3 and trying again
s

Safouane Chergui

11/17/2022, 10:30 AM
and just to clarify something, the param first_param is added dynamically to
extra_params
in
register_config_loader
f

FlorianGD

11/17/2022, 10:31 AM
So,
kedro run --pipeline my_pipeline --params first_param:first_value
is not what you actually run?
s

Safouane Chergui

11/17/2022, 10:32 AM
Well, adding it to extra_params in register_config_loader is equivalent to passing to in kedro run as a parameter
or so I thought 😅
f

FlorianGD

11/17/2022, 10:33 AM
Could you try and see if it changes something?
s

Safouane Chergui

11/17/2022, 10:34 AM
It works that way... So, they are different after all 😅
f

FlorianGD

11/17/2022, 10:35 AM
We are more and more trying to have static pipelines, rather than adding too much dynamically, except for parameters, and we do not have this problem. We do use a custom ConfigLoader, but this is to use the parameters in the catalog, it should not change the way parameters are passed to nodes
👍 1
s

Safouane Chergui

11/17/2022, 10:47 AM
Well, I’ve just tested both options and
extra_params
in
register_config_loader
has the same content whether it’s added dynamically or through kedro run. But as the error says that it didn’t find the parameter in the data catalog, I think it should be injected in it as well So, it becomes a big hurdle just to avoid passing to kedro run directly 😅
f

FlorianGD

11/17/2022, 10:56 AM
Frankly, as
register_config_loader
is removed in Kedro
0.18+
, I would not go down this road
👍 1
s

Safouane Chergui

11/17/2022, 10:58 AM
You’re right 👍 Thanks for your help 👍
👍 1
4 Views