https://kedro.org/ logo
Join Slack
Powered by
Hi everyone ! I think I may have already ask abou...
# questions
m
Hi everyone ! I think I may have already ask about this previously… But can’t find it back. Since we use the same repo / pipeline for multiple clients, we have organized things in this way
Copy code
conf/
├── client_A
│   ├── base
│   └── local
└── client_B
    ├── base
    └── local
This works just fine with 
kedro run --conf-source=
But i’m under the impression that this breaks 
kedro ipython
Even if we hack things around by hardcoding the conf-source in 
settings.py
we always end with the following error: 
ValueError: Given configuration path either does not exist or is not a valid directory: /workspaces/new-ingestion/conf/base
 This is really strange, since nowhere did we pass such a path… It looks like kedro ipython is making the “hard assumption” that this is where things are supposed to be… In case it’s helpful, here’s the traceback:
Copy code
⬢ [Docker] ❯ kedro ipython
ipython --ext kedro.ipython
Python 3.10.7 (main, Jul 18 2023, 08:54:35) [GCC 12.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.17.2 -- An enhanced Interactive Python. Type '?' for help.
[11/03/23 09:35:39] INFO     Resolved project path as: /workspaces/new-ingestion.                                                                         __init__.py:139
                             To set a different path, run '%reload_kedro <project_root>'                                                                                 
[TerminalIPythonApp] WARNING | Error in loading extension: kedro.ipython
Check your config files in /home/vscode/.ipython/profile_default
Traceback (most recent call last):
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/IPython/core/shellapp.py", line 282, in init_extensions
    self.shell.extension_manager.load_extension(ext)
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/IPython/core/extensions.py", line 76, in load_extension
    return self._load_extension(module_str)
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/IPython/core/extensions.py", line 93, in _load_extension
    if self._call_load_ipython_extension(mod):
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/IPython/core/extensions.py", line 145, in _call_load_ipython_extension
    mod.load_ipython_extension(self.shell)
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/kedro/ipython/__init__.py", line 46, in load_ipython_extension
    reload_kedro()
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/kedro/ipython/__init__.py", line 93, in reload_kedro
    session = KedroSession.create(
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/kedro/framework/session/session.py", line 195, in create
    session._setup_logging()
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/kedro/framework/session/session.py", line 213, in _setup_logging
    logging_config = self._get_logging_config()
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/kedro/framework/session/session.py", line 202, in _get_logging_config
    logging_config = self._get_config_loader()["logging"]
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/kedro/config/config.py", line 117, in __getitem__
    return self.get(*self.config_patterns[key])
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/kedro/config/config.py", line 131, in get
    return _get_config_from_patterns(
  File "/home/vscode/.pyenv/versions/3.10.7/envs/kedro_dodo_ingestion/lib/python3.10/site-packages/kedro/config/common.py", line 73, in _get_config_from_patterns
    raise ValueError(
ValueError: Given configuration path either does not exist or is not a valid directory: /workspaces/new-ingestion/conf/base
N.B: we’ve experimented with 
reload_kedro(env=xxx)
, with no success so far…
n
Can u share the output running the tree command?
j
a
We just merged this in https://github.com/kedro-org/kedro/pull/3199. Turns out 
kedro ipython --conf-source=<conf>
is a little bit harder than expected but you should be able to do 
%reload_kedro --conf-source=<new_conf>
this 1
n
Thank you both, was just looking for the PR
m
Excellent ! News. Thanks everyone 🙏🏼 🙂
a
Although I think starting 
kedro ipython
or 
%load_ext kedro.ipython
will still give the error because the kedro extension still assumes the normal 
base/
and 
local/
conf structure. Maybe having a dummy standard conf folder will help, and then you can reload kedro with the new conf path with the line magic?
Open in Slack
PreviousNext
Powered by