https://kedro.org/ logo
#questions
Title
# questions
p

Paul Mora

11/27/2023, 10:21 AM
Hey team, I have a questions regarding technology usage.
Namely, has anybody ever used
VertexAI
in combination with
kedro
and have some experiences to share? I am not sure whether if we are dealing with a plain Vanilla DS use-case that the usage of pure
AutoML
from
Vertex
wouldn’t be advantageous compared to the inclusion of
kedro
m

marrrcin

11/27/2023, 10:30 AM
We’ve been using and maitaining Kedro Vertex AI plugin: https://github.com/getindata/kedro-vertexai Main reason why we combine Kedro with Vertex AI Pipelines: • with Kedro you can easily build pipelines fast and run them reliably locally • with Vertex AI you can scale up the pipeline to larger data / larger machines, everything is serverless • it’s easier to write pipelines in Kedro than in KFP SDK, especially because KFP SDK is more low-level And some others. When it comes to other Vertex AI services, they are less related to Kedro, e.g. Auto ML can be used regardless of other technologies and it has different focus.
p

Paul Mora

11/27/2023, 10:39 AM
Sounds great, thanks for the answer 🙂 ! Stupid questions: 1. I understand that writing custom pipelines in
VertexAI
would be done through KFP SDK which is more complicated than
kedro
pipelines, therefore
kedro
is the nicer choice - is that correct? 2. Why would you say that the
AutoML
has a different focus? For a straight plan Vanilla DS use-case, the
AutoML
would seem like an okay option. Is there something I am missing?
2. Probably the
AutoML
is more for deep-learning I assume?
m

marrrcin

11/27/2023, 10:41 AM
1. Yes, that’s my opinion 😉 2. AutoML has different focus than VAI Pipelines / Kedro. AutoML is for building models / automating the DS stuff. VAI Pipelines / Kedro are focused on building ML pipelines (so more end-to-end).
p

Paul Mora

11/27/2023, 10:42 AM
Got it! Thanks 🙂
3 Views