Hey guys, Is there any way we can pass in a runtime argument into the catalog? kedro docs explicitly state that "`runtime_params` are not designed to override

globals

configuration.", so I wonder if there's any workaround to my requirement: We have a project where the frontend action will trigger my kedro run in Databricks (via Databricks Jobs REST API). Some parameters from the frontend will override some of my default kedro parameters (this works fine), but i also need to override a dataset definition based on one of these parameters. Particularly, i want my dataset to be written to a specific location that depends on a runtime_param `simulation_id`: my globals.yaml:

root: ${oc.env:AWS_S3_ROOT}
simulation_id: ${uuid:""} # ideally something like ${runtime_params:simulation_id}, but i know it's not possible
folders:
    m_frontend: "09_frontend_reporting/${..simulation_id}"

my catalog.yaml:

simulation_json:
  type: json.JSONDataset
  filepath: ${globals:root}/${globals:folders.m_frontend}/simulation_${globals:simulation_id}.json

What are my options to achieve this?

hi @Pedro Sousa Silva, not sure if it helps but there's a way to combine globals and runtime_params: https://docs.kedro.org/en/latest/configuration/advanced_configuration.html#how-to-use-globals-and-runtime-params

model_options:
  random_state: "${runtime_params:random, ${globals:my_global_value}}"

which is nice because it provides a way to specify the params, and the defaults are given by

globals.yml

would this address your use case?

thanks @Juan Luis, but that's for parameters only, though, right? i'm interested in overriding a catalog entry

oh but you can define a variable and then use it

wow, can't believe i missed that. i've been using that the whole time for stuff like:

_dbx_con: &dbx_con
  type: <dbx_dataset>
  catalog: <my_catalog>

mytable:
  <<: *dbx_con
  database: bronze_sanitized
  table: mytable

but never made the connection that this would work for a single variable as well. let me test it!

this should work with "normal" YAML syntax btw! the resolvers are Omegaconf-specific

it works ofc - thanks for the callout!