Hi everyone 👋 I am happy to share a new plugin I have been working on for a few months: kedro-azureml-pipeline, which lets you run, schedule and monitor Kedro pipelines on Azure ML Pipelines without changing your pipelines, catalog or hooks. It started as a fork of kedro-azureml by GetInData (thank you!), which is unfortunately no longer actively maintained. The core approach is unchanged, but the config schema, the CLI and the docs have been rewritten, and a lot of features have been added. It has been running in production at Evolta Technologies, and I'm grateful to them for making this release possible 🙏 What's new compared to kedro-azureml • Named jobs in config: each way of running your project is declared under

jobs

in

azureml.yml

and submitted with

kedro azureml run -j <job>

. Jobs are reviewable, version-controlled and reused across ad-hoc runs, schedules and CI. • Scheduling: cron and recurrence triggers live in config and deploy with

kedro azureml schedule -j <job>

. • Multiple workspaces: define dev / staging / prod side by side and target one with

-w

. • Job factories: if your pipelines are namespaced per variant, write one templated job and let the concrete jobs be derived from the namespaces. • Verify without touching Azure: stale config keys fail loudly by name, and

kedro azureml compile --check

compiles every job in memory with no credentials and nothing submitted, so you can run it on every PR. There's also

run --dry-run

. • Full hook lifecycle: all Kedro hooks fire during remote execution, and it works with kedro-mlflow (now on MLflow 3). • Simpler data passing: Azure ML native pipeline data passing only, so no temporary blob storage account to set up. Intermediate datasets not in your catalog still flow between steps automatically. Get started • Step by step migration guide: It's mostly a package swap, a find-and-replace in your catalog, and a rewrite of

azureml.yml

. • Docs: kedro-azureml-pipeline.readthedocs.io • Repo: github.com/stateful-y/kedro-azureml-pipeline •

pip install kedro-azureml-pipeline

(or

kedro-azureml-pipeline[mlflow]

) I'd love your help & feedback It would mean a lot if you could: • Try it on one of your projects, especially if you're currently on kedro-azureml • Report bugs, rough edges in the migration, or docs gaps • Share how you run Kedro on Azure ML today and what's missing • Reach out if you'd like to contribute! Looking forward to hearing what you think! :)

That's great news @Guillaume Tauzin!! 🤩 Thank you so much for creating and releasing this plugin! 👌

This is amazing!!! Thanks so much @Guillaume Tauzin 🚀🎉