What can cause `conf_source` in omegaconfigloader ...
# questions
What can cause
in omegaconfigloader to duplicate project name? I've encountered the error before but don't remember the cause. self.context.config_loader.conf_source = '/home/adobrogo/projects/kedro/kedro-vertexai/spaceflights/spaceflights/conf' Answer: When migrating to 0.19 check your
which version of kedro are you on? and how
is defined in
its not defined, version 0.19.3
where is the project root? i.e. where is your
it's spaceflights-pandas template - it's in root dir of spaceflights
there are two
in the path you provided, is it the outer or the inner one?
this is the path it looks for, when I run debug mode I've read the conf_source param of
and that's its value that is incorrect. I only have 1 spaceflights in the path
Oh interesting, does
kedro run
or any kedro project command works as expected? is it just this attribute looks suspicious
yes it does...
the problem arises when I try to manually load/patch config loader
Looks fine for me, actually , what is
? Is this your own code or coming from kedro
Copy code
cl: AbstractConfigLoader = self.context.config_loader
            if self.CONFIG_KEY not in cl.config_patterns.keys():
                        self.CONFIG_KEY: [
            vertex_conf = self._ensure_obj_is_dict(
I remember the cause before was when someone mixed accidently templated config loader with omegaconf. I think I've swapped all instances of TemplatedConfig class to OmegaConf, but maybe not
^ this code throws error
nevermind, I'll keep looking for that
Is this a subclass of ConfigLoader? this isn't coming from Kedro so it's hard for me to determine what's going on
it's likely an incorrect implementation
yes I've probably missed something to update
trying to bump kedro-vertex to kedro 0.19
šŸ‘€ 1
maybe @marrrcin know something about this?
I doubt it he remembers šŸ˜„, probably would need to dig in as well
Yeah, so this code was to handle cases during Kedro's transition from ConfigLoader/TemplatedConfigLoader into OmegaConf. Where does it throw?
Copy code
[03/21/24 13:47:29] WARNING  Failed to confirm consent. No data was sent to Heap. Exception: [Errno 2] No such file or directory:                                             plugin.py:306
ā•­ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ Traceback (most recent call last) ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā•®
ā”‚ /home/adobrogo/projects/kedro/virtualenv/lib/python3.10/site-packages/kedro_vertexai/context_hel ā”‚
ā”‚ per.py:66 in config                                                                              ā”‚
ā”‚                                                                                                  ā”‚
ā”‚   63 ā”‚   ā”‚   ā”‚   ā”‚   ā”‚   }                                                                       ā”‚
ā”‚   64 ā”‚   ā”‚   ā”‚   ā”‚   )                                                                           ā”‚
ā”‚   65 ā”‚   ā”‚   ā”‚   vertex_conf = self._ensure_obj_is_dict(                                         ā”‚
ā”‚ ā± 66 ā”‚   ā”‚   ā”‚   ā”‚   self.context.config_loader.get(self.CONFIG_KEY)                             ā”‚
ā”‚   67 ā”‚   ā”‚   ā”‚   )                                                                               ā”‚
ā”‚   68 ā”‚   ā”‚   except MissingConfigException:                                                      ā”‚
ā”‚   69 ā”‚   ā”‚   ā”‚   if not isinstance(cl, OmegaConfigLoader):                                       ā”‚
ā”‚                                                                                                  ā”‚
ā”‚ /home/adobrogo/.pyenv/versions/3.10.4/lib/python3.10/_collections_abc.py:819 in get              ā”‚
ā”‚                                                                                                  ā”‚
ā”‚    816 ā”‚   def get(self, key, default=None):                                                     ā”‚
ā”‚    817 ā”‚   ā”‚   'D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.'                    ā”‚
ā”‚    818 ā”‚   ā”‚   try:                                                                              ā”‚
ā”‚ ā±  819 ā”‚   ā”‚   ā”‚   return self[key]                                                              ā”‚
ā”‚    820 ā”‚   ā”‚   except KeyError:                                                                  ā”‚
ā”‚    821 ā”‚   ā”‚   ā”‚   return default                                                                ā”‚
ā”‚    822                                                                                           ā”‚
ā”‚                                                                                                  ā”‚
ā”‚ /home/adobrogo/projects/kedro/virtualenv/lib/python3.10/site-packages/kedro/config/omegaconf_con ā”‚
ā”‚ fig.py:199 in __getitem__                                                                        ā”‚
ā”‚                                                                                                  ā”‚
ā”‚   196 ā”‚   ā”‚   else:                                                                              ā”‚
ā”‚   197 ā”‚   ā”‚   ā”‚   base_path = str(Path(<http://self._fs.ls|self._fs.ls>("", detail=False)[-1]) / self.base_env)       ā”‚
ā”‚   198 ā”‚   ā”‚   try:                                                                               ā”‚
ā”‚ ā± 199 ā”‚   ā”‚   ā”‚   base_config = self.load_and_merge_dir_config(  # type: ignore[no-untyped-cal   ā”‚
ā”‚   200 ā”‚   ā”‚   ā”‚   ā”‚   base_path, patterns, key, processed_files, read_environment_variables      ā”‚
ā”‚   201 ā”‚   ā”‚   ā”‚   )                                                                              ā”‚
ā”‚   202 ā”‚   ā”‚   except UnsupportedInterpolationType as exc:                                        ā”‚
ā”‚                                                                                                  ā”‚
ā”‚ /home/adobrogo/projects/kedro/virtualenv/lib/python3.10/site-packages/kedro/config/omegaconf_con ā”‚
ā”‚ fig.py:286 in load_and_merge_dir_config                                                          ā”‚
ā”‚                                                                                                  ā”‚
ā”‚   283 ā”‚   ā”‚   # noqa: too-many-locals                                                            ā”‚
ā”‚   284 ā”‚   ā”‚                                                                                      ā”‚
ā”‚   285 ā”‚   ā”‚   if not self._fs.isdir(Path(conf_path).as_posix()):                                 ā”‚
ā”‚ ā± 286 ā”‚   ā”‚   ā”‚   raise MissingConfigException(                                                  ā”‚
ā”‚   287 ā”‚   ā”‚   ā”‚   ā”‚   f"Given configuration path either does not exist "                         ā”‚
ā”‚   288 ā”‚   ā”‚   ā”‚   ā”‚   f"or is not a valid directory: {conf_path}"                                ā”‚
ā”‚   289 ā”‚   ā”‚   ā”‚   )                                                                              ā”‚
MissingConfigException: Given configuration path either does not exist or is not a valid directory: /home/adobrogo/projects/kedro/kedro-vertexai/spaceflights/spaceflights/conf/base
I noticed that it got it wrong even at the telemetry warning before
I think it was working before... I'm very puzzled. kedro run/kedro ipython get it right
this happens only when I try to
kedro vertexai run-once
probably some context-loading helper class does something in way that was changed but I can't track it down yet
yea I'm not sure how to debug it though
maybe I can link it instead packaging and installing and then try to use debugger
Yup, that's the most convenient way - just remember to use poetry in both places
I hate configuring debugger in vs code šŸ˜„
šŸ¤£ 1
pycharm ftw
> I hate configuring debugger in vs code šŸ˜„ (edited) Same here, but recently I learnt it's less annoying than it used to be. At least for the case of Kedro I always have to look up the path of the executable before. Now you can choose
Python module
and add args, then it's done. This is equivalent to
python -m kedro <args>
(i.e. args = ["run"] 99% of the time)
So FYI @Nok Lam Chan the issue was change in API for
šŸ‘šŸ¼ 1
šŸŽ–ļø 1