Hello 👋 , Is it possible to use OmegaConf nested interpolation in the catalog? I would like to do something like:

"{table}_ds":
  type: my_custom_ds
  site: "${globals:site}"
  file_id: "${globals:file_id[${table}]}"

and in

globals.yml

file_id:
   users: ""
   items: ""

site: ""

Thanks!

Hey Justin, it’s not possible to do this currently. The

globals:

resolver is resolved when the config is loaded before a session is run. The

{table}

placeholder from the dataset factory gets filled in later when a pipeline is being executed.

okay thanks. Any idea how I can achieve a two step interpolation like I need?

I’ve added your use case to this ticket. I don’t think there is a straighforward way to achieve this other than using the file-id in the dataset name as a placeholder.

okay thx