Hello everyone, I do have a question on pipeline r...
# questions
Hello everyone, I do have a question on pipeline registry, how can I specify the order of the pipeline to execute?
Or better question is how do I connect the disconnected pipelines on pipeline__registry.py
Can you explain what you are trying to do? It may help to understand the context better since these 2 questions are quite different
I have 3 different pipelines created using
kedro pipeline create
. Those pipelines are disconnected since it doesn't contain the output from the previous nodes. So what, I'm trying to do is to register those 3 different pipelines on the
and connect each pipeline after another pipeline
I see - so you just want to run all pipelines together right?
You don’t need to do anything special, pipelines can be disconnected. In your
, you can add
pipelines["all"] = pipelines["a"] + pipelines["b"] + pipelines"[c"]
(or whatever name you preferred)
You can then run
kedro viz
to confirm it - they should be just 3 parallel pipelines
for example, pipeline["b"] should only run once pipeline["a"] is done running
and pipeline["c"] will run only after pipeline["b"] is finished
what does these pipeline do and why it has to be run after another pipeline without any explicit dependencies?
the execution order is determined by Node dependencies, that is a node has some input and output, Kedro then try to resolve this graph. If you need to guarantee this order, then your last node of pipeline_a should produce something that the first node of pipeline_b would take.
Thanks for the response