Hi everyone. I'm trying to deploy my kedro pipelin...
# plugins-integrationss
Sebastian Cardona Lozano
01/12/2023, 11:22 PMHi everyone. I'm trying to deploy my kedro pipeline in Vertex AI in GCP using the kedro-vertexai plugin, but I have problems with the installation. When I run
kedro info Copy code
WARNING Failed to load kedro_vertexai.cli commands from EntryPoint(name='vertexai', value='kedro_vertexai.cli:commands', group='kedro.project_commands'). Full exception: utils.py:338
invalid syntax (generator.py, line 191)
[01/12/23 23:05:27] WARNING Failed to load kedro_vertexai.cli commands from EntryPoint(name='vertexai', value='kedro_vertexai.cli:commands', group='kedro.project_commands'). Full exception: utils.py:338
invalid syntax (generator.py, line 191)
_ _
| | _____ __| |_ __ ___
| |/ / _ \/ _` | '__/ _ \
| < __/ (_| | | | (_) |
|_|\_\___|\__,_|_| \___/
v0.18.4
Kedro is a Python framework for
creating reproducible, maintainable
and modular data science code.
Installed plugins:
kedro_docker: 0.3.1 (entry points:project)
kedro_telemetry: 0.2.3 (entry points:cli_hooks,hooks)
kedro_vertexai: 0.6.0 (entry points:hooks,project)
kedro_viz: 5.1.1 (entry points:global,line_magic)e
em-pe
01/12/2023, 11:39 PMcould you try with the newer kedro-vertexai plugin version? the most recent one is
0.8.1☝️ 1
✅ 1
s
Sebastian Cardona Lozano
01/13/2023, 9:07 PMHi @em-pe. Thanks for your advice. Effectively I tried with version 0.8.1 and Python 3.9 (I was working with the default version 3.7) and it worked and created the pipeline in Vertex AI. Now I have another issue, because when the pipeline starts to run in Vertex it rised the following error:
Copy code
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /usr/local/bin/kedro:8 in <module> │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/framework/cli/cli.py:211 in │
│ main │
│ │
│ 208 │ """ │
│ 209 │ _init_plugins() │
│ 210 │ cli_collection = KedroCLI(project_path=Path.cwd()) │
│ ❱ 211 │ cli_collection() │
│ 212 │
│ │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1130 in __call__ │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/framework/cli/cli.py:139 in │
│ main │
│ │
│ 136 │ │ ) │
│ 137 │ │ │
│ 138 │ │ try: │
│ ❱ 139 │ │ │ super().main( │
│ 140 │ │ │ │ args=args, │
│ 141 │ │ │ │ prog_name=prog_name, │
│ 142 │ │ │ │ complete_var=complete_var, │
│ │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1055 in main │
│ │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1657 in invoke │
│ │
│ /usr/local/lib/python3.9/site-packages/click/core.py:1404 in invoke │
│ │
│ /usr/local/lib/python3.9/site-packages/click/core.py:760 in invoke │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/framework/cli/project.py:381 in │
│ run │
│ │
│ 378 │ tag = _get_values_as_tuple(tag) if tag else tag │
│ 379 │ node_names = _get_values_as_tuple(node_names) if node_names else n │
│ 380 │ │
│ ❱ 381 │ with KedroSession.create(env=env, extra_params=params) as session: │
│ 382 │ │ session.run( │
│ 383 │ │ │ tags=tag, │
│ 384 │ │ │ runner=runner(is_async=is_async), │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/framework/session/session.py:18 │
│ 0 in create │
│ │
│ 177 │ │ session._store.update(session_data) │
│ 178 │ │ │
│ 179 │ │ # we need a ConfigLoader registered in order to be able to set │
│ ❱ 180 │ │ session._setup_logging() │
│ 181 │ │ return session │
│ 182 │ │
│ 183 │ def _get_logging_config(self) -> Dict[str, Any]: │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/framework/session/session.py:19 │
│ 5 in _setup_logging │
│ │
│ 192 │ def _setup_logging(self) -> None: │
│ 193 │ │ """Register logging specified in logging directory.""" │
│ 194 │ │ try: │
│ ❱ 195 │ │ │ logging_config = self._get_logging_config() │
│ 196 │ │ except MissingConfigException: │
│ 197 │ │ │ self._logger.debug( │
│ 198 │ │ │ │ "No project logging configuration loaded; " │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/framework/session/session.py:18 │
│ 4 in _get_logging_config │
│ │
│ 181 │ │ return session │
│ 182 │ │
│ 183 │ def _get_logging_config(self) -> Dict[str, Any]: │
│ ❱ 184 │ │ logging_config = self._get_config_loader()["logging"] │
│ 185 │ │ # turn relative paths in logging config into absolute path │
│ 186 │ │ # before initialising loggers │
│ 187 │ │ logging_config = _convert_paths_to_absolute_posix( │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/config/config.py:111 in │
│ __getitem__ │
│ │
│ 108 │ │ ) │
│ 109 │ │
│ 110 │ def __getitem__(self, key): │
│ ❱ 111 │ │ return self.get(*self.config_patterns[key]) │
│ 112 │ │
│ 113 │ def __repr__(self): # pragma: no cover │
│ 114 │ │ return ( │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/config/config.py:125 in get │
│ │
│ 122 │ │ return _remove_duplicates(self._build_conf_paths()) │
│ 123 │ │
│ 124 │ def get(self, *patterns: str) -> Dict[str, Any]: # type: ignore │
│ ❱ 125 │ │ return _get_config_from_patterns( │
│ 126 │ │ │ conf_paths=self.conf_paths, patterns=list(patterns) │
│ 127 │ │ ) │
│ 128 │
│ │
│ /usr/local/lib/python3.9/site-packages/kedro/config/common.py:69 in │
│ _get_config_from_patterns │
│ │
│ 66 │ │
│ 67 │ for conf_path in conf_paths: │
│ 68 │ │ if not Path(conf_path).is_dir(): │
│ ❱ 69 │ │ │ raise ValueError( │
│ 70 │ │ │ │ f"Given configuration path either does not exist " │
│ 71 │ │ │ │ f"or is not a valid directory: {conf_path}" │
│ 72 │ │ │ ) │
╰──────────────────────────────────────────────────────────────────────────────╯
ValueError: Given configuration path either does not exist or is not a valid
directory: /home/kedro_docker/conf/localdocker runkedro docker runm
marrrcin
01/13/2023, 11:36 PMSeems like the conf/local is in .dockerignore, either run in a different Kedro environment or add those files to the docker image
s
Sebastian Cardona Lozano
01/16/2023, 8:47 PMHi @marrrcin/@em-pe. Thanks for your time. You were right, I had to add the
conf/localloge
em-pe
01/16/2023, 9:05 PMIt’s actually done on purpose - by convention
conf/localkedro runconf/baseconf/<your-env>👍 1
152 Views