Kedro is an open-sourced Python framework for creating maintainable and modular data science code.

Kedro

image.png

turns out that, according to <https://ethical.institute/state-of-ml-2024|this survey> (N=200+), almost 40 % of model serving is just a FastAPI or Flask wrapper, 18 % is "None", and 17 % is custom built :grimacing: that's 75 %

similarly, for ETL / workflow orchestration, 28 % is Airflow, 20 % is None, and 14 % is custom built = 62 %

there's no data cleaning in the survey (!) but looks like Argo goes next, followed closely by Databricks, then Prefect followed closely by Kubeflow

it's a good reminder that

• GitHub stars != adoption
• happy users exist != mindshare

I think the fact that Argo Workflows is so high is weird.

Then you see that the most common respondents are Staff+ MLEs. I feel these answers are better for figuring out, how do people design their ML platforms.