Hi! I am using OmegaConfigLoader. I am using 2 e...
# questions
e
Hi! I am using OmegaConfigLoader. I am using 2 environments: 1.
base
with a catalog_globals.yml:
_env: dev
2.
databricks_qas
with a catalog_globals.yml:
_env: qas
In my base folder I have a catalog.yml
Copy code
pidata_silver:
  type: projetc.io.databricks.ManagedTableDataSet
  database: xx_aa_${_env}_silver
  table: operations_concentrator_pidata_tpidata
  write_mode: upsert
  primary_key: timestamp_str
  dataframe_type: spark
  partition_columns: year_month
  layer: silver
However when I use the conf
databricks_qas
the variable in my catalog
xx_aa_${_env}_silver,
is not changed to
xx_aa_qas_silver
how I was expecting. It keeps
xx_aa_dev_silver
, it seems that in the catalog there is no override. Is this true? any workaround? @Kremlin Huaman
if I define again
pidata_silver,
I can override the one in base. Not ideal, because I need to duplicate catalog. It does not make a lot of sense to be able to use
_env
as “global” inside the same enviroment, since at the end of the day I need to duplicate the catalog anyways.