Hi Team, can I add an optional node to pipeline, It should be executed only if specific parameter is set?

You can do a

NOOP

if the parameter value is null for example, but the pipeline needs to be static.

Will that create problem while running?, like if a parameter is set to

True

, I want to run the node and get a dataframe for example and if its

False

, I want to move to next node without any output

If you plan to output something from this node, then it will show some challenges

This is one of our teams questions regarding kedro, I am not exactly aware of why they need this

When it comes to Kedro itself, pipelines need to be static DAGs. You can work-around some of the issues (like here, with the pseudo-conditional node, but it’s not natively supported right now).

Got it, thanks

I think @marrrcin is right here. To supplement, a static DAG is preferred due to simplicity and reproducibility. If you think about pipeline visualisation, when you have multiple switches within nodes, it is very difficult to reason the execution logic (it is almost like a hidden graph within your node). Especially when you have a large computation graph. So even when you can do this, it’s still best to avoid this. We do recognize in some cases it’s unavoidable. As of the moment, the approach suggested by @marrrcin is quite common