Let's say I have two subpipelines A and B. Subpipeline A contains a node (call it

node_x

) whose output is used by subpipeline B. Subpipeline B contains a node (call it

node_y

) whose output is not dependent on subpipeline A or any other node. When I

kedro run

I think but am not positive that

node_y

can run at any time including before

node_x

. Is that correct? The reason I'm not positive is that subpipeline B has the output of

node_x

as input. So if the inputs to the subpipeline are evaluated before the nodes are executed, it follows that no node in subpipeline B can run without

node_x

completing first.

Provided

node_y

doesn't depend anything from other node, it can be executed before ALL THE NODES OF subpipeline A are executed. Note that subpipeline/pipelines does not exist once they are merged into a graph, internally Kedro only see node dependencies, that in turn form the DAGs.

@Marshall Krassenstein I wanted to reply but Nok was faster 🙂 Yeah there is no notion of subpipeline in Kedro,

kedro run

always runs a single pipeline. If what you call subpipelines are then summed up, from the Kedro perspective it's a single DAG.

Thank you both!