I have another question. Is it possible to link the catalog and params? Currently I am using a globals file from which params is sourcing itself and the catalog too because one of the names of the files used in the pipeline contains a globals variable. I would like to get rid of the globals file and only use parameters because support runtime parameters.

Yes, that should be possible. Here's the relevant documentation that explains how to do this using the `OmegaConfigLoader`: https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-override-configuration-with-runtime-parameters-with-the-omegaconfigloader

Ok thank you very much !

@Juan Luis another proof that people want the runtime_params resolver in globals.yml!

yeah our params are quite confusing in general...

Hello, I tried using custom resolvers as mentionned in the article. I declared a s3resolver function in settings.py to access it in my production and development catalog. The problem is that I keep having the same error: ValueError: Duplicate keys found in path to my production catalog and in path to my development catalog : … In fact these catalog entries have the same name but refer to different files in different s3 buckets. The loading of these prameters works well but I get an error. I thought maybe it came from the fact that I instanciate a kedrosession below in the settings.py file to retreive the credentials and the parameters using OmegaConfigLoader. Does anyone know how could I get rid of this error ?

Looping in @Dmitry Sorokin who is the wizard this week

Ok thanks!

Hi Armand, could you elaborate a bit more on what you’re trying to do with the resolver in the

settings.py

?

Hello, thanks for your answer. I am loading a flat file from s3 to retreive two variables : LAST_PERIOD and ANALYZED_QUARTERS. I load them in my resolver to access them in my catalog. In fact, some names of my catalog are dynamic and rely on the value of these variables that are loaded from s3.

What is happening with the

global_settings

variable after? I think it’s the creation of the

KedroSession

that is causing some confusion with the environments. The custom resolvers are meant to be defined in the

settings.py

but then used in the catalog/parameters yaml files. https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-use-resolvers-in-the-omegaconfigloader Maybe the creation of the session and the loading of the config in

settings.py

is not needed?

Nothing is happening to the globals_settings variable in the file, I only use it to store the variables to acess them in my nodes. I create a session because I want to store the credentials (and parameters) in variables to acess them in my nodes. I don’t know how could I do without doint this

I would suggest not creating a new session in the settings. Ideally, credentials shouldn’t directly be accessed in the nodes but if you have to, you could try using the

oc.env

resolver into your parameters https://docs.kedro.org/en/0.19.14/configuration/advanced_configuration.html#how-to-load-credentials-through-environment-variables You’d have to activate the

oc.env

resolver in your settings.py

from omegaconf.resolvers import oc

CONFIG_LOADER_ARGS = {
    "base_env": "base",
    "default_run_env": "local",
    "custom_resolvers": {
        "oc.env": oc.env,
    }
}

Thank you very much for tour answer! I appreciate it