Sergey S11/16/2023, 11:22 AM
). Is there an automatic way of combining two different pipelines with different and separate requirements when converting a Kedro pipeline into a workflow orchestration pipeline (i.e. Kubeflow Pipeline, Prefect, Airflow, etc) ?
Juan Luis11/16/2023, 1:11 PM
datajoely11/16/2023, 1:16 PM
Sergey S11/16/2023, 1:16 PM
Juan Luis11/20/2023, 9:47 AM
• A modular pipeline might have external dependencies specified in a localfile.
• Pipeline specific dependencies are scooped up during the micro-packaging process.https://docs.kedro.org/en/stable/nodes_and_pipelines/modular_pipelines.html#providing-modular-pipeline-specific-dependencies we have a pending task to revisit our micro-pipeline flow https://github.com/kedro-org/kedro/milestone/21 we'd love to hear your thoughts when you have more experience with it @Sergey S