Not to necro an old thread, but does OmegaConf help with this? Specifically: defining a list of constants in

settings.py

and looping over it in the jinja-esque style to define catalog entries. Couldn’t immediately tell from the docs, though I haven’t had much time to work with the new loader. I am excited too of course 🙂

Thanks for your reply! It kind of does, but imo, it's not very straightforward, and I don't want too many vars in global 😕

When you get a sec, do you have a minimal example? I’m not seeing how to do a for loop in the OmegaConf docs

I can’t think of a good solution now. Do you have an example of the Catalog and things that you are trying to loop for? They don’t have to be real, just trying to get a sense of the use case.

@Ian Whalen Could resolvers help with this?

For

TemplatedConfigLoader

in

settings.py

I would (want to) do:

fruit_list = ["apple", "banana", "pear"]

CONFIG_LOADER_CLASS = TemplatedConfigLoader
CONFIG_LOADER_ARGS = {
    "globals_pattern": "*globals.yml",
    "globals_dict": {"fruit": fruit_list},
}

and then in my

catalog.yml

do:

{% for fruit in fruit_list %}

{{ fruit }}.raw_data:
    type: pandas.CSVDataset
    filepath: data/01_raw/{{ fruit }}.csv

{% endfor %}

Then as you can imagine I can also use my

settings.fruit_list

elsewhere to generate modular pipelines. This is what I wanted in the original thread I linked.

globals_dict

doesn’t work like that for

TemplatedConfigLoader

. Unless something changed. I noticed a note about

ac_context

in the release notes but didn’t see a change to the

TemplatedConfigLoader

signature.