Hi all, I want to ask about the Kedro Azureml plugin, 1. Is there any way to deploy a specific pipeline into the Azure pipeline? I have multiple pipelines in my Kedro project, and as far as I know, this plugin will change the pipeline into a Job in the Azure pipeline. Let's say I have pipeline A and pipeline B, when Kedro Azureml init, is there any argument to pass only a specific pipeline? 2. I just want to make sure that Kedro Azureml need compute clusters, right? Or it could be deployed in compute instance? Thanks all

1. You can deploy any pipeline from Kedro, just use

kedro azureml run --pipeline=<your pipeline>

2. It needs compute cluster, yes. You can also check out full quickstart and video tutorial here https://kedro-azureml.readthedocs.io/en/stable/source/03_quickstart.html

Regarding question number 2, is it possible to use the Kubernetes cluster in Azure ML Workspace? I have AKS and want to utilize it.

We haven’t tested this scenario yet. Feel free to test it and raise an issue on GitHub if it doesnt

Alright, thank you so much for helping 🙌

If you want to use Azure pipelines, then you have to use compute clusters. If you want to use compute instances (which are basically notebooks), you can run there without the plugin, just with

kedro run

as you would do locally

Hi! when deploying a pipeline in azure pipeline, what would be the best solution to schedule lets say in daily basis? Having alarms, retries, etc. In my CI/CD I was thinking in using the plugin to create the yaml of the pipeline, then terraform to deploy resources and use an azure activity from adf pointing to this pipeline? does this make sense for you?

What about using Azure DevOps for that? You could build the docker image with Kedro project and then invoke it using

kedro azureml run

from the subsequent step in the same pipeline

hi, @marrrcin about this question to change the compute target into compute instances. Is it possible to change a few lines of client.py so that it could run in compute instances? I have tried and it works to run in compute instances by changing this lines.

Sure, it could be made configurable - we’re keen to accept a PR for that 🙂