https://kedro.org/ logo
#questions
Title
# questions
s

Sarthak Bhatt

01/05/2024, 8:40 AM
Hi - How should i reference values from parameter_data_processing.yml in my pipeline in kedro? i have multiple parameters file in my project
d

datajoely

01/05/2024, 10:00 AM
the file name is not important, the key name is
s

Sarthak Bhatt

01/05/2024, 1:46 PM
understood. thanks!
also can you tell me how can i pass the entire parameters_data_processing.yml file in the pipeline inputs?
d

datajoely

01/05/2024, 2:36 PM
so
if they are top level keys you need to be explicit
but if they are nested you can pass the whole thing
Copy code
my_key: 1,2,3

my_nested_key: 
    my_sub_key: 1
    my_other_sub_key: 2
you can explicitly provide
my_nested_key
and it will pass a dictionary of
Dict[Dict[str, int]]
s

Sarthak Bhatt

01/05/2024, 2:39 PM
understood. thanks a ton!
d

datajoely

01/05/2024, 2:39 PM
my pleasure
s

Sarthak Bhatt

01/05/2024, 2:45 PM
@datajoely - in my case in one of my node im passing parameters as input. parameters.yml contains 4 top keys. there are other parameters file also like parameters_datapreprocessing.yml. how passing only parameters is able to link with the keys inside the parameters.yml file only?
d

datajoely

01/05/2024, 3:10 PM
so you should just name them explicitly
Copy code
inputs=["time_split_app_proc", "parmam:key1", "param:key2", ...]
s

Sarthak Bhatt

01/05/2024, 3:14 PM
yes but my question is when i pass parameters why only keys from parameters.yml are passed and not from parameters_dataprocessing.yml? Is it matching with the file name?
d

datajoely

01/05/2024, 3:25 PM
you should get an error stating they need to be unique
the parameter naemspace doesn’t take files into account, the files are for humans to make it easier to manage
there are more complex ways of having multiple versions of the same key too but I’m not sure that’s relevant here
s

Sarthak Bhatt

01/05/2024, 4:24 PM
The keys are unique in both yml. But keys from parameters.yml is only accessed inside node wheb input = “parameters” . Can i similarly access parameters_processing.yml with input = “parameters_processing”? Is that correct way?
d

datajoely

01/05/2024, 4:25 PM
the filename is irrelevant, they’re all scooped up into the same list of available keys
if you send
parameters
that sends all parameters across all files
you can use dot syntax to send explicit key paths
param:my_key.my_subkey
s

Sarthak Bhatt

01/05/2024, 4:43 PM
Clear! Thanks again! 🙌
~hi @datajoely - can you help me in resolving this error? ValueError: Pipeline input(s) {'parameters_data_insights'} not found in the DataCatalog Below are screenshots of yml and input in pipeline~
I want to pass all the parameters under the key parameters_data_insights from the parameters_data_insights.yml