https://kedro.org/ logo
#plugins-integrations
Title
# plugins-integrations
g

George Drakoulas

03/19/2024, 7:52 AM
Hey team, I would like to ask if there is any tutorial that combines kedro, kedro-azureml, kedro-mlflow, and kedro-airflow. In general it would be super useful, to create one for a simple ML training, testing and inference pipeline including deployment in azure and pre-processing steps
šŸ‘€ 2
šŸ’„ 1
j

Juan Luis

03/19/2024, 8:54 AM
@Nok Lam Chan and myself are getting my feet wet with MLFlow and @Dmitry Sorokin and @Ankita Katiyar have worked a lot with Airflow in the past weeks. I proposed talk and workshop proposals for various Python conferences along those lines, so there's a chance they get accepted in any case, we want to produce more educational content combining different pieces of the mlops stack, so stay tuned
šŸ„³ 2
m

marrrcin

03/19/2024, 9:08 AM
Why do you need airflow and azureml at the same time?
šŸ’” 2
a

Artur Dobrogowski

03/19/2024, 1:12 PM
all of these introduce different concepts and integrations, all of them have their own quickstarts/tutorials how to use them. What's the benefit of having this particular set of configurations? Why not kedro-kubeflow, kedro-vertexai or kedro-sagemaker? There's too many possibilities imho, but a spaceflights-like starter for each plugin would be probably helpful or maybe all-in-one big cookie cutter where you can pick and choose (as they all are pretty independent with their configs that should be doable), but that should not be responsibility of kedro devs imho šŸ˜›
j

Juan Luis

03/19/2024, 1:25 PM
(assuming @Artur Dobrogowskiā€™s question was directed to me) there's no special benefit in this particular set of configurations, it happens to be more popular as far as I understand
d

Deepyaman Datta

03/19/2024, 3:50 PM
Why not kedro-kubeflow, kedro-vertexai or kedro-sagemaker? There's too many possibilities imho
Which is why you need a common abstraction to support universal deployment. šŸ˜ (Not an unreasonable idea IMO, which is why you've had projects like https://github.com/couler-proj/couler, but nothing being actively developed/worked on at this time AFAIK.)
a

Artur Dobrogowski

03/19/2024, 3:54 PM
it was meant as a rethorical question on why it's not feasible to make such a starter, as there's going to be someone that asks for a different set of plugins, unless it's one that's universal for most of them
well kedro-mlflow is tangential to the execution environment ones
šŸ‘šŸ¼ 1
šŸ‘ 1
2 Views