Hi Team, Currently we’re using

kedro==0.18.3

that pins

pytest~=6.2

which is conflict with

pandera

, a new dependency we want to introduce.

kedro==0.18.5

has already

pytest~=7.2

, so we’re not far from resolving this conflict. On the other hand, in order to upgrade to a higher Kedro version, we would need to change our custom

JinjaTemplatedConfigLoader

that inherits from

AbstractConfigLoader

, as both

0.18.4

and

0.18.5

introduced changes to configuration management, the latter

OmegaConf

specifically. Also,

0.18.6

fixes some regressions in

0.18.5

. Question: based on the above context, what Kedro would you suggest for us to upgrade to? • It seems at least we need to go to

0.18.6

but maybe we can aim all the way to

0.18.10

? • Also, do you have a migration guide about how to migrate from a custom config loader to OmegaConf bearing in mind that we need to use

multi-runner

as well? Thank you! @Kasper Janehag, @Jaskaran Singh Sidana

hey @Mate Scharnitzky! if your

JinjaTemplatedConfigLoader

works with 0.18.3 but not with 0.18.6 or any other later

0.18.x

, we'd love to know more. it's definitely not the intention to introduce breaking changes before major releases

It definitely caused some issues for us, but since we relaxed/cleaned a lot of our dependencies. Let me try to move the pin to

0.18.10

and I’m happy to report back the issues we identify.

I don’t expect any breaking changes for any 0.18.x release other than the dependencies issue.

Hi @Nok Lam Chan, @Juan Luis As we discussed, we created a PR to upgrade Kedro to

0.18.10

. After resolving a few dependency conflicts, all of our kedro pipelines’ integration tests fail with the below error:

│ /home/circleci/.pyenv/versions/3.9.17/lib/python3.9/collections/__init__.py: │
│ 1058 in __getitem__                                                          │
│                                                                              │
│   1055 │   │   │   return self.data[key]                                     │
│   1056 │   │   if hasattr(self.__class__, "__missing__"):                    │
│   1057 │   │   │   return self.__class__.__missing__(self, key)              │
│ ❱ 1058 │   │   raise KeyError(key)                                           │
│   1059 │                                                                     │
│   1060 │   def __setitem__(self, key, item):                                 │
│   1061 │   │   self.data[key] = item                                         │
╰───────────────���──────────────────────────────────────────────────────────────╯
KeyError: 'logging'

this rings a bell 😅 https://kedro-org.slack.com/archives/C03RKP2LW64/p1683732820562689

Exactly 🙂 , full circle

I guess

logging

wasn’t available? Check if you logging files are actually there maybe?

but it was working before right? it's a change that we've done midway

Is it running with some kedro starters?

we need to get to the bottom of this I think. I'm opening an issue

It runs the kedro pipelines for our propensity use case which includes: • the data mocker pipeline • the feature engineering pipeline • and finally the propensity pipeline

I guess the error shows up very early before any of these pipelines get run?

yes, the mocker doesn’t even complete, I assume if I typed

kedro registry list

would fail as well

can you do

$ tree conf/

? to see what config files you have

I’m building it locally now to be able to test it

so 0.18.3 is still working and 0.18.10 breaks it. Do you know since version it starts to break?

correct

Most likely is this PR https://github.com/kedro-org/kedro/commit/997692a22da1a67f220b3c4cf8e3935076d28e11#diff-02bfe6b42015e13642cd09c4f0[…]4eb5c9d47c35a51b2d5c9be1ad00c01, but I am surprise if this breaks anything

I managed to replicate the error locally

we found the root cause, updating soon