Airflow is far from dead! <
# resources
Personal Opinion I think Airflow is a wonderful piece of technology and has adapted to the K8s / cloud native world in a way that Luigi, Azkaban and others around the same time could not. If I was starting a company tomorrow I would reach for Flyte (🥈), Meatflow (🥉 )or Dagster (🥇) because they are data-centric and optimised for ML workflows by design.
💡 1
oh I see @Elijah ben Izzy is co-author!
K 1
Or if you’re Airflow 2.0 you’re a new kid on the block

^ the part at 32:49 is super relevant to Kedro cc @Ankita Katiyar
yup, we got to chat a good amount with Jarek at PyCon US, great guy!
Yeah — fun one to write! So Hamilton/Kedro + dlt/dbt (all good asset layer) options can work nicely with an orchestrator. Also using an orchestrator as a scheduler thus leveraging dagster/prefect/airflow/metaflow/flyte to run those/track larger jobs (personaly I’d use dagster/prefect for ease of iteration).
🔥 2
(conflict in interest, because it’s written by a competitor, but it’s amusing nonetheless)