Hello everyone, I do have a question on pipeline r...
# questionse
Eluard Camota
10/27/2023, 1:48 PMHello everyone, I do have a question on pipeline registry, how can I specify the order of the pipeline to execute?
Eluard Camota
10/27/2023, 1:51 PMOr better question is how do I connect the disconnected pipelines on pipeline__registry.py
n
Nok Lam Chan
10/27/2023, 2:00 PMCan you explain what you are trying to do? It may help to understand the context better since these 2 questions are quite different
e
Eluard Camota
10/27/2023, 2:18 PMI have 3 different pipelines created using
kedro pipeline createpipeline_registry.pyn
Nok Lam Chan
10/27/2023, 2:27 PMI see - so you just want to run all pipelines together right?
Nok Lam Chan
10/27/2023, 2:28 PMYou don’t need to do anything special, pipelines can be disconnected. In your
pipeline_registry.pypipelines["all"] = pipelines["a"] + pipelines["b"] + pipelines"[c"]Nok Lam Chan
10/27/2023, 2:28 PMYou can then run
kedro vize
Eluard Camota
10/27/2023, 2:35 PMfor example, pipeline["b"] should only run once pipeline["a"] is done running
Eluard Camota
10/27/2023, 2:35 PMand pipeline["c"] will run only after pipeline["b"] is finished
n
Nok Lam Chan
10/27/2023, 4:07 PMwhat does these pipeline do and why it has to be run after another pipeline without any explicit dependencies?
Nok Lam Chan
10/27/2023, 4:09 PMthe execution order is determined by Node dependencies, that is a node has some input and output, Kedro then try to resolve this graph. If you need to guarantee this order, then your last node of pipeline_a should produce something that the first node of pipeline_b would take.
e
Eluard Camota
10/27/2023, 4:23 PMThanks for the response
3 Views