Hello ! I got this error when deploying pipeline via vertexai :

<http://com.google.cloud.ai|com.google.cloud.ai>.platform.common.errors.AiPlatformException: code=RESOURCE_EXHAUSTED, message=The following quota metrics exceed quota limits: <http://aiplatform.googleapis.com/custom_model_training_cpus|aiplatform.googleapis.com/custom_model_training_cpus>, cause=null; Failed to create custom job for the task. Task: Project number: 496232377396, Job id: 1189445081858310144, Task id: 6444159035313750016, Task name: preprocess-shuttles-node, Task state: DRIVER_SUCCEEDED, Execution name: projects/496232377396/locations/europe-west1/metadataStores/default/executions/14295685814278275726; Failed to create external task or refresh its state. Task:Project number: 496232377396, Job id: 1189445081858310144, Task id: 6444159035313750016, Task name: preprocess-shuttles-node, Task state: DRIVER_SUCCEEDED, Execution name: projects/496232377396/locations/europe-west1/metadataStores/default/executions/14295685814278275726; Failed to handle the pipeline task. Task: Project number: 496232377396, Job id: 1189445081858310144, Task id: 6444159035313750016, Task name: preprocess-shuttles-node, Task state: DRIVER_SUCCEEDED, Execution name: projects/496232377396/locations/europe-west1/metadataStores/default/executions/14295685814278275726

I check the quotas specified but it's not the problem because it's set to 1 and I specify 0.2 cpus for each node (kedro vertexai starter guide). I think it come from gcp but i know know witch configuration to update. Someone has an explaination / face the same bug ? I'm on this issue for days and i can't find the solution...

hi @Adrien, you're referring to https://kedro-vertexai.readthedocs.io/en/stable/source/02_installation/02_configuration.html am I right? could you share the relevant configuration? cc @marrrcin

I’ts a problem in your GCP project - the error is self explanatory - quota is not enough (you’re probably on 0). Check https://stackoverflow.com/q/73368320/1955346 Keep in mind that if your quota is at 1, you’re not guaranteed to run the pipeline even if you set requirements in vertexai.yml to 0.2, as underneath the hood, GCP stil needs to spin up a virtual machine, which most likely will have > 1 cores.

Thanks you for your answers, indeed it come from gcp quotas limitation. I can't increase it by myself... Curently speaking with the support. It really sucks... They are slow to react