Hello everyone, we used Kedro to build a pipeline that validates datasets. The pipeline has grown a lot and sometimes we need validations to run in a certain order. I know that Kedro decides order based on node inputs and outputs and in the past we created dummy datasets to create dependencies but this makes the code harder to read and maintain. We also use Apache Airflow and in airflow dependencies are defined explicitly between nodes by a custom operator: Node1 >> Node2 >> Node3 This sounds like a problem that must have been encountered before so I wanted to ask around whether there already is a plugin or extension that provides this functionality?

Hi Jan, There isn't a dedicated plugin or extension that provides Airflow-style

>>

operators for explicit dependencies. There's an old GitHub issue (#1156) from 2022 requesting an

additional_upstream

parameter for explicit GitHub dependencies, but this hasn't been implemented.

Thank you. I found that issue and the problem is exactly what I was talking about. The suggested solution does not help me because the nodes that need to run in order have a logical dependency but do not access the same datasets. I found a more up to date discussion https://github.com/kedro-org/kedro/discussions/3758 about the topic and will add my request there.