https://kedro.org/ logo
Join Slack
Powered by
hi folks, are custom omegaconf resolvers supposed ...
# questions
j
hi folks, are custom omegaconf resolvers supposed to work for the catalog? I’m trying to define one but the raw 
${name:value}
strings are passed to the dataset. I can elaborate more if needed
m
AFAIK resolvers are only enabled for credentials
j
that thread looks familiar 😄
there is an example in the docs using custom resolvers for parameters https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-use-custom-resolvers-in-the-omegaconfigloader but I can’t seem to make it work for catalog
apparently it’s the 
oc.env
resolver the one that’s only enabled for credentials
🤔 1
m
SchrödingersConfigLoader
😄
sadcat2 2
So it’s either custom or none
j
but then it’s my understanding that custom ones should work right?
m
My understanding is that custom will work if you extend 
OmegaConfigLoader
and use it in settings.py
n
My understanding is oc.env enabled for Credentials. Custom resolver is not possible with the built-in config loader. There is an open issue but we haven’t started working on it. Current solution should be a custom omegaconfigloader, the documentation have a section describing how to extend it.
I expect the custom resolver would work for any config, it should be generic so Catalog & parameters should work
👍🏼 1
If it’s not the case I can have a look
j
thanks both! I'll give this one last chance, and open a bug if it doesn't work
👍🏼 1
okay, I was being hit by one of the sub-issues of https://github.com/kedro-org/kedro/issues/2458 (probably https://github.com/kedro-org/kedro/issues/2466) which @Nok Lam Chan fixed recently
n
Are you on the latest branch? Both of them should be merged in 
main
j
yes, on 
main
everything works fine!
n
Sweet!
j
by the way, I think I found a trivial way of registering custom resolvers:
Copy code
# settings.py

if not OmegaConf.has_resolver("dummy"):
    OmegaConf.register_new_resolver("dummy", lambda x: x)

CONFIG_LOADER_CLASS = OmegaConfigLoader
see my comment on https://github.com/kedro-org/kedro/issues/2407#issuecomment-1549682252
n
You are absolutely right. You can always do dynamic patching, I used it a lot even for developing pipeline in a notebook
I didn’t know this hasn’t been added to the backlog, I would love to have it earlier because it’s one of the value-added functions with OmegaConf. I don’t see there are too many controversies, I guess we just need to come up with the syntax (I quite like @datajoely’s idea). Maybe we can discuss this in next week’s Tech Design?
K 1
37 Views
Open in Slack
PreviousNext
Powered by