Hi Team, I want to make use of templating for the catalog. My idea is to define the catalog in

conf/base

and then the respective parameters in each environment, like

conf/dev

,

conf/local

, etc. I imaging something like this

# conf/base/catalog.yaml

companies:
  type: pandas.CSVDataset
  filepath: {_raw}/companies.csv

# conf/local/catalog_globals.yaml

_raw: data/raw

#conf/dev/catalog_globals.yaml

_raw: cloud-path/raw

The problem is, that it only works if I have the

catalog.yaml

in each of the environments, which means a lot of duplication for me. Is there a better way I could do that?

Hi @Viktoriia, which kedro version are you using

0.19.11

Did you try this - https://docs.kedro.org/en/stable/configuration/configuration_basics.html#how-to-specify-additional-configuration-environments

yes, that's exactly what I'm trying

okay let me try on my side and will get back to you

Your project loads

conf/base/

as the bottom-level configuration environment but allows you to overwrite it with any other environments that you create

As I understand it, kedro will first read my catalog and the template values from the

conf/base.

But if I don't have the copy of the catalog itself (only the template values) in

conf/dev

, the catalog itself won't be overwritten

Hi @Viktoriia, You can try using globals - https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-use-global-variables-with-the-omegaconfigloader Make sure the globals file is named as

globals.yml

and not

globals.yaml

. Thanks for helping @Ankita Katiyar

it works, thanks so much 🙂