https://kedro.org/ logo
Powered by
#questions
Title
# questions
j

Juan Luis

05/16/2023, 7:53 AM
hi folks, are custom omegaconf resolvers supposed to work for the catalog? I’m trying to define one but the raw 
${name:value}
strings are passed to the dataset. I can elaborate more if needed
m

marrrcin

05/16/2023, 8:04 AM
AFAIK resolvers are only enabled for credentials
j

Juan Luis

05/16/2023, 8:17 AM
that thread looks familiar 😄
there is an example in the docs using custom resolvers for parameters https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-use-custom-resolvers-in-the-omegaconfigloader but I can’t seem to make it work for catalog
apparently it’s the 
oc.env
resolver the one that’s only enabled for credentials
🤔 1
m

marrrcin

05/16/2023, 9:42 AM
SchrödingersConfigLoader
😄
sadcat2 2
So it’s either custom or none
j

Juan Luis

05/16/2023, 11:03 AM
but then it’s my understanding that custom ones should work right?
m

marrrcin

05/16/2023, 11:14 AM
My understanding is that custom will work if you extend 
OmegaConfigLoader
and use it in settings.py
n

Nok Lam Chan

05/16/2023, 11:59 AM
My understanding is oc.env enabled for Credentials. Custom resolver is not possible with the built-in config loader. There is an open issue but we haven’t started working on it. Current solution should be a custom omegaconfigloader, the documentation have a section describing how to extend it.
I expect the custom resolver would work for any config, it should be generic so Catalog & parameters should work
👍🏼 1
If it’s not the case I can have a look
j

Juan Luis

05/16/2023, 11:59 AM
thanks both! I'll give this one last chance, and open a bug if it doesn't work
👍🏼 1
okay, I was being hit by one of the sub-issues of https://github.com/kedro-org/kedro/issues/2458 (probably https://github.com/kedro-org/kedro/issues/2466) which @Nok Lam Chan fixed recently
n

Nok Lam Chan

05/16/2023, 12:53 PM
Are you on the latest branch? Both of them should be merged in 
main
j

Juan Luis

05/16/2023, 12:54 PM
yes, on 
main
everything works fine!
n

Nok Lam Chan

05/16/2023, 12:57 PM
Sweet!
j

Juan Luis

05/16/2023, 1:31 PM
by the way, I think I found a trivial way of registering custom resolvers:
Copy code
# settings.py

if not OmegaConf.has_resolver("dummy"):
    OmegaConf.register_new_resolver("dummy", lambda x: x)

CONFIG_LOADER_CLASS = OmegaConfigLoader
see my comment on https://github.com/kedro-org/kedro/issues/2407#issuecomment-1549682252
n

Nok Lam Chan

05/16/2023, 1:41 PM
You are absolutely right. You can always do dynamic patching, I used it a lot even for developing pipeline in a notebook
I didn’t know this hasn’t been added to the backlog, I would love to have it earlier because it’s one of the value-added functions with OmegaConf. I don’t see there are too many controversies, I guess we just need to come up with the syntax (I quite like @datajoely’s idea). Maybe we can discuss this in next week’s Tech Design?
K 1
3 Views
Powered by