Hello guys, is it possible to use a custom runner with kedro Vertex ai ? I know the plugin has his own runner but maybe their is a way to use both ?

The

VertexAIPipelinesRunner

looks quite simple, so you could extend it or include it's functionality in your custom runner or use that? There would be no native way to combine runners.

Thanks for your answer ! I wanted to do that but when vertexai compile the pipeline, on each node he is calling the default Vertexai runner. So if I do that, it will not work on vertexai cmoud

I have not used the plugin, but I would say you should be able to patch the command to use your runner.

By forking vertexai plugin ? It was my last solution ...

You can override Click commands in your project. It's not 100% straightforward, but doable

Mmmh I'll think about it thanks :) not really used to click command

If you want to override a Click group, you need to import that group, and then you can write new commands to overwrite it. Most likely, you will import stuff from the VertexAI plugin, so you're not rewriting (much) code.

Yes, if there is an easier way to do it would be perfect 👌 Seems a little bit complicated

We're keen to accept a PR for that feature 😉 What use case you want to cover by overriding the runner ?

I want to implement pytorch cluster computing with vertexai. I need to make it work fast and then i'll make a PR similar to kedro-azureml

If you are really in a hurry then… 🙈

from unittest.mock import patch
with patch('kedro_vertexai.vertex_ai.runner.VertexAIPipelinesRunner', new=YourRunner):
    # do magic here :D

Wow your line seems magic :0 I run this in the cli.py ?