https://kedro.org/ logo
#questions
Title
# questions
d

Declan Jackson

02/19/2024, 11:18 AM
Hi team, how can we define the same data set in a catalog file across multiple environments? I have a two
catalog.yml
files save like so:
Copy code
conf/base/catalog.yml
conf/test/catalog.yml
In each file I have the same datasets but with different types, filepaths, etc. and when I try to run my pipelines with
kedro run --env=test
I get the following error:
Copy code
ValueError: Duplicate keys found in [...]/conf/test/catalog.yml and [...]/conf/base/catalog.yml: input_files@{type}
It says in the docs here that duplicate keys should be resolved with OmegaConfigLoader, but am I missing something? Thanks 🙂 (btw I am running Kedro 0.19.2)
K 1
d

datajoely

02/19/2024, 11:22 AM
within one env you have duplicate keys
I think
d

Declan Jackson

02/19/2024, 11:24 AM
Even when I updated the two
catalog.yml
files to only have one data set entry each the error still occurs
d

datajoely

02/19/2024, 11:25 AM
hmm that doesn’t sound right 🤔
d

Declan Jackson

02/19/2024, 11:25 AM
The error is even comparing the files across environments
d

datajoely

02/19/2024, 11:25 AM
can you share more of the stack trace?
d

Declan Jackson

02/19/2024, 11:33 AM
ah looking at the stack trace helped me figure it out...the error was caused by a hook trying to load in the catalog incorrectly - thanks!
d

datajoely

02/19/2024, 11:33 AM
🙏
ty 1