Hi everyone I m trying to deploy my kedro pipeline in Vertex

Question

Hi everyone I m trying to deploy my kedro pipeline in Vertex AI in GCP using the kedro vertexai plugin but I have problems with the installation When I run `kedro info` in the CLI I got this error at the beginning and I can t use any command of this plugin ```WARNING Failed to load kedro vertexai cli commands from EntryPoint name= vertexai value= kedro vertexai cli commands group= kedro project commands Full exception utils py 338 invalid syntax generator py line 191 01 12 23 23 05 27 WARNING Failed to load kedro vertexai cli commands from EntryPoint name= vertexai value= kedro vertexai cli commands group= kedro project commands Full exception utils py 338 invalid syntax generator py line 191 | | | | | | ` | | lt | | | | | | | | | | v0 18 4 Kedro is a Python framework for creating reproducible maintainable and modular data science code Installed plugins kedro docker 0 3 1 entry points project kedro telemetry 0 2 3 entry points cli hooks hooks kedro vertexai 0 6 0 entry points hooks project kedro viz 5 1 1 entry points global line magic ``` I m working on a Vertex AI workbench wich has Anaconda and Jupyter I d appreciate your help to solve this issue Thanks PD kedro docker and kedro viz work perfect < C03RKP2LW64|questions> < C03RKPCLYGY|plugins integrations>

em-pe · Answer

could you try with the newer kedro vertexai plugin version the most recent one is `0 8 1` gt <https pypi org project kedro vertexai >

Sebastian Cardona Lozano · Answer

Hi < em pe> Thanks for your advice Effectively I tried with version 0 8 1 and Python 3 9 I was working with the default version 3 7 and it worked and created the pipeline in Vertex AI Now I have another issue because when the pipeline starts to run in Vertex it rised the following error ``` Traceback most recent call last usr local bin kedro 8 in lt module gt usr local lib python3 9 site packages kedro framework cli cli py 211 in main 208 209 init plugins 210 cli collection = KedroCLI project path=Path cwd 211 cli collection 212 usr local lib python3 9 site packages click core py 1130 in call usr local lib python3 9 site packages kedro framework cli cli py 139 in main 136 137 138 try 139 super main 140 args=args 141 prog name=prog name 142 complete var=complete var usr local lib python3 9 site packages click core py 1055 in main usr local lib python3 9 site packages click core py 1657 in invoke usr local lib python3 9 site packages click core py 1404 in invoke usr local lib python3 9 site packages click core py 760 in invoke usr local lib python3 9 site packages kedro framework cli project py 381 in run 378 tag = get values as tuple tag if tag else tag 379 node names = get values as tuple node names if node names else n 380 381 with KedroSession create env=env extra params=params as session 382 session run 383 tags=tag 384 runner=runner is async=is async usr local lib python3 9 site packages kedro framework session session py 18 0 in create 177 session store update session data 178 179 we need a ConfigLoader registered in order to be able to set 180 session setup logging 181 return session 182 183 def get logging config self gt Dict str Any usr local lib python3 9 site packages kedro framework session session py 19 5 in setup logging 192 def setup logging self gt None 193 Register logging specified in logging directory 194 try 195 logging config = self get logging config 196 except MissingConfigException 197 self logger debug 198 No project logging configuration loaded usr local lib python3 9 site packages kedro framework session session py 18 4 in get logging config 181 return session 182 183 def get logging config self gt Dict str Any 184 logging config = self get config loader logging 185 turn relative paths in logging config into absolute path 186 before initialising loggers 187 logging config = convert paths to absolute posix usr local lib python3 9 site packages kedro config config py 111 in getitem 108 109 110 def getitem self key 111 return self get self config patterns key 112 113 def repr self pragma no cover 114 return usr local lib python3 9 site packages kedro config config py 125 in get 122 return remove duplicates self build conf paths 123 124 def get self patterns str gt Dict str Any type ignore 125 return get config from patterns 126 conf paths=self conf paths patterns=list patterns 127 128 usr local lib python3 9 site packages kedro config common py 69 in get config from patterns 66 67 for conf path in conf paths 68 if not Path conf path is dir 69 raise ValueError 70 f Given configuration path either does not exist 71 f or is not a valid directory conf path 72 ValueError Given configuration path either does not exist or is not a valid directory home kedro docker conf local``` This error also raises when I run the container with `docker run` in the CLI in the local machine Nevertheless when I run it with `kedro docker run` It runs perfectly I m following the quick start guide Thanks again < C03RKP2LW64|questions> < C03RKPCLYGY|plugins integrations>

marrrcin · Answer

Seems like the conf local is in dockerignore either run in a different Kedro environment or add those files to the docker image

Sebastian Cardona Lozano · Answer

Hi < marrrcin> < em pe> Thanks for your time You were right I had to add the `conf local` and `log` folders to the Docker image although kedro vertex didn t do On the other hand maybe do you have an example or a more detailed explanation of the vertexai yml to configure the resources for specific nodes or to send notifications

em-pe · Answer

It s actually done on purpose by convention `conf local` should contain only the configuration that you are using on your local machine with `kedro run` The basic config should land under `conf base` which is the default when you don t specify environment or in `conf lt your env gt ` if you want to use the custom one When it comes to the documentation I believe all we have at the moment are the comments that land in the generated default config file see <https github com getindata kedro vertexai blob 9c07518af6476cdcb87dba2304027b52d733096e kedro vertexai config py L7>