https://kedro.org/ logo
Join Slack
Powered by
Hi team, I am using OmegaConfigLoader to load my p...
# questions
m
Hi team, I am using OmegaConfigLoader to load my parameters yml files. One variable is pointing to globals.yml but for some reasons the conf loader is only reading parameters and not globals.yml. Do you know how I can make it read both ?
Copy code
conf_loader = OmegaConfigLoader(
        conf_source=CONF_SOURCE, env="base"
    )
params = conf_loader.get("parameters", "globals")
n
Are you using it in code or in a Kedro project?
"parameters" and "globals" are two separate keys and you won't find 
globals
in the parameters dictionary. This is why in the YAML config you use the 
$globals:
resolver. If you need to access the 
globals
dictionary directly for some reason, you should do 
conf_loader["globals"]
Can you explains what you are trying to do?
m
I am using it in a kedro project. So the reason why I am trying to do so is that in my create_pipeline function, I'll need to loop on my params file to execute a nide multiple times with different params. So I need to load the params in the create_pipeline function
It is part of a code of an already developped asset. They were using TemplateConfigLoader and it was working for them. Now with OmegaConfigLoader it is not working anymore..
But if you have an easier way to load the yaml params in create_pipeline without that it would be even better
Screenshot 2024-07-17 at 18.18.33.png
and params = conf_loader.get("parameters", "globals")
n
How did you get the config loader?
this looks more like a custom implementation and it's hard to give an answer. If you have the config loader already, you can access global with 
conf_loader["globals"]
m
conf_loader = OmegaConfigLoader( conf_source=CONF_SOURCE, env="base" )
n
More broadly this is creating pipeline on the fly. These "pipeline parameters" are quite different from the parameters that get passed into the node. If possible, I'd separate these parameters to a separate file and simply load it with 
yaml.load
as most likely you don't need any config loader feature. It looks like you are just iterating on a list or something
m
Yes i am iterating on a list in a parametrs.yml file but some attrbute ate $VAR where VAR is defined in globals.yaml ..
the loader is not able to read it and is saying var is unknown
n
conf_loader["globals"]
can you use this instead?
m
will it also load parameters?
this is the strcuture of my file
my params I need are in transactions.yml
and they are referring to variables in globals.yml
n
Can you try to print out the parameters and see?
Copy code
conf_loader = OmegaConfigLoader(
        conf_source=CONF_SOURCE, base_env="base", default_run_env="local"
    )
I think you just have the env wrong
m
when I do print conf_loader["globals"] I get my params
n
m
I'll try !
same error...
10 Views
Open in Slack
PreviousNext
Powered by