Hello. Newbie here. Pls whats the best practice fo...
# questions
o
Hello. Newbie here. Pls whats the best practice for using kedro with airflow ( astro ). Should i ; 1. create 2 seperate projects ( astro and the kedro ) and then move the kedro project files into the airflow project ( where exactly do i put them? ) 2. create the airflow project and develop the kedro project within it.
y
I have a follow up question for this. Are you asking this because you want to merge the two project templates? I know Kedro has one, but so does Astro.
o
Hi. Well, kinda. i'm asking cos i am to develop my first project with kedro and intend to use airflow for orchestration. i have seen that both kedro and airflow have templates but just thinking of the most optimal way of going about development ( seperately then merge OR together in the same project).
j
hi @olufemi george, I just checked and I would recommend that you 1. create the Kedro project, 2.
cd
to the project directory, and 3. do
astro dev init --from-template learning-airflow
the reason is that
astro dev init
works in the current directory, whereas
kedro new
does not. alternatively, you can 1. create a new directory, 2. do
astro dev init --from-template learning-airflow
3. initialise a new Python library using
uv init --lib
4. use my unofficial
kedro-init
plugin
uvx kedro-init .
does this make sense?
d
hi @olufemi george, we have some recommendations how to work with astro+kedro here: https://docs.kedro.org/en/0.19.14/deployment/airflow.html#deployment-process-with-astro-cli
🥳 1