Hi Team! I am upgrading to `kedro>=0.19`, but ...
# questions
a
Hi Team! I am upgrading to
kedro>=0.19
, but struggling with getting my catalog and parameters being discovered by
OmegaConfigLoader
The folder structure for
parameters
is something like this (same for
catalog)
Copy code
conf/base/
└── parameters/
    ├── <usecase1>/
    │   └── usecase1.yml
    └── <usecase2>/
        ├── <usecase2a>/
        │   ├── usecase2a_1.yml
        │   └── usecase2a_2.yml
        └── <usecase2b>/
            └── usecase2b.yml
This used to work with
kedro<0.19
and
OmegaConfigLoader
, but now I am really struggling with how to set the glob pattern for: 1. Discovering catalog in any folder depth as long as they are under a folder named
catalog
2. Discovering parameters in any folder depth as long as they are under a folder named
parameters
3. Discovering catalog globals (where to place them?) 4. Discovering parameter globals (where to place them?) Thanks! 🙂
K 1
m
I don't think we changed anything on how config is discovered with
OmegaConfigLoader
between pre and post
0.19.0
, (right @Ankita Katiyar? ). Can you share the error you're seeing?
a
With the default settings, this folder structure should work ^
a
It's surprising that the somehow the same pattern works perfectly in
kedro<0.18
but breaks in
kedro>=0.19
a
What error are you getting? And what are you passing to
CONFIG_LOADER_ARGS
in
settings.py
?
Might be this https://docs.kedro.org/en/stable/resources/migration.html#changes-to-the-default-environments You would also need to pass
base_env
and
default_run_env
in
CONFIG_LOADER_ARGS
along with the other things you were passing 🤔
👍 1
🥳 1
a
Omg! Thankyou @Ankita Katiyar thankyou I think this did it but let me confirm it since another issue popped up. But seems like this is it!
❤️ 1
K 1
Confirmed working! Thanks again 🙂
👍 2
For everyone's benefit, here are the list of things I did for the kedro 19 migration: - [x] Upgrade kedro version to
kedro>=0.19.0
- [x] Add dependency
kedro_dataset>=3.0.0
- [x] Move kedro dataset imports to
kedro_datasets
- [x] DataSet to Dataset in catalog - [x] DataSet to Dataset when importing - [x] Move layer to metadata.layer in catalog - [x] Update settings.py since now
OmegaConfigLoader
is default - [x] Clean node tags to contain only letters, digits, hyphens, underscores and/or fullstops.