Hi team, need help with kedro-airflow integration. There are 2 edgenodes(think of VM) , 1 shared edgenode and 1 application edgenode. The kedro project resides in the app edgenode and has the all the dependencies installed, its been triggered from the airflow from the main shared edgenode. After creating the dag file (consider as sub dag) for the kedro pipe and trying to trigger from the airflow main DAG in the shared VM, get a ’ModuleNotFoundError: No module found ‘kedro’. looks like it implies Kedro be installed in the shared edgenode as well. Is that the case or is there any alternate ways to not install Kedro in the main one. The conda env to process and execute DAGs across these two edgenodes are separate. Can the KedroOperator be made to use the kedro and its dependencies installed in where it resides?

@Ivan Danov Would you be able to help here?

The conda env to process and execute DAGs across these two edgenodes are separate. Can the KedroOperator be made to use the kedro and its dependencies installed in where it resides? (edited)

I don’t have a clear answer yet but you can probably use

PythonVirtualenvOperator

or something similar to achieve this. In this case, at least the application node needs to have kedro installed. A potentially related issue. https://github.com/kedro-org/kedro-plugins/issues/27