Hey team, I have a questions regarding technology usage.

Namely, has anybody ever used

VertexAI

in combination with

kedro

and have some experiences to share? I am not sure whether if we are dealing with a plain Vanilla DS use-case that the usage of pure

AutoML

from

Vertex

wouldn’t be advantageous compared to the inclusion of

kedro

We’ve been using and maitaining Kedro Vertex AI plugin: https://github.com/getindata/kedro-vertexai Main reason why we combine Kedro with Vertex AI Pipelines: • with Kedro you can easily build pipelines fast and run them reliably locally • with Vertex AI you can scale up the pipeline to larger data / larger machines, everything is serverless • it’s easier to write pipelines in Kedro than in KFP SDK, especially because KFP SDK is more low-level And some others. When it comes to other Vertex AI services, they are less related to Kedro, e.g. Auto ML can be used regardless of other technologies and it has different focus.

Sounds great, thanks for the answer 🙂 ! Stupid questions: 1. I understand that writing custom pipelines in

VertexAI

would be done through KFP SDK which is more complicated than

kedro

pipelines, therefore

kedro

is the nicer choice - is that correct? 2. Why would you say that the

AutoML

has a different focus? For a straight plan Vanilla DS use-case, the

AutoML

would seem like an okay option. Is there something I am missing?

1. Yes, that’s my opinion 😉 2. AutoML has different focus than VAI Pipelines / Kedro. AutoML is for building models / automating the DS stuff. VAI Pipelines / Kedro are focused on building ML pipelines (so more end-to-end).

Got it! Thanks 🙂