Hi, we are migrating to v1.0 some old v.18 pipelines and I have a question about parameters.yml and runtime_params. (we run into an issue if not receiving a runtime_params:begin_date) We reference runtime_params in catalog, as an example

...
filepath: "${globals:example_bucket}/i/j/k/date=${runtime_params:begin_date}/

only the base/parameters.yml has a definition for begin_date

begin_date: "2025-01-01"

and the CONFIG_LOADER_ARGS (in settings.py) does not have an entry for config_patterns. (stepping with a debugger in OmegaConfigLoader constructor initialization confirms the “parameters” patterns are present). and we invoke the pipeline as no params (kedro run). The error we receive when catalog loads is:

InterpolationResolutionError: Runtime parameter 'begin_date' not found and no default value provided.

Thank you!

can you try providing a default value ? Like - filepath: "${globals:example_bucket}/i/j/k/date=${runtime_params:begin_date, "2025-01-01"}/ or provide a param with

kedro run

. I am not sure if the resolution of runtime parameters is made strict with kedro 1.0, but this should work. Also make sure the runtime_params resolver is used outside of globals config type and to override configuration at runtime as mentioned here - https://docs.kedro.org/en/1.0.0/configure/advanced_configuration/#how-to-override-configuration-with-runtime-parameters-with-the-omegaconfigloader Thank you

> can you try providing a default value ? specifying a default value fixes crashing, however the values from parrameters.yml still don’t show (uses default value provided) > Also make sure the runtime_params resolver is used outside of globals config I’ve read that link but I really don’t understand what needs to be done. If you can provide an example I would really appreciate it!

runtime_params

resolver is used to resolve parameters that are provided at runtime. If you want to share variables across configuration types, you can use globals as mentioned here - https://docs.kedro.org/en/1.0.0/configure/config_loader_migration/#7-globals

runtime_params

resolver is used to resolve parameters that are provided at runtime.

Does the above applies only to the CLI

--params

argument? Or does it include parameters.yml as well? (I’m debugging / setting breakpoints in OmegaConfLoader__getitem__ and by the time catalog loads it looks runtime_params were not loaded yet (only globals, catalog yml)

you can use globals as mentioned here …

But the globals cannot reference runtime_params /parameters.yml (I get “The

runtime_params:

resolver is not supported for globals.” exception)? (my goal is to add begin_date, end_date to the files referenced in catalog - how 0.18 worked).

What's the problem here? If you define it in

base

you can reference it using

${begin_date}

?

Hi, (see OP) for an issue we run into; the parameters.yml:begin_date is not visible during catalog reading so the interpolation fails. If we pass

--params begin_date=...

those are visible. While only later in the pipeline nodes execution Context class does merger parameters.yml and CLI params.

I wish I have an updated version of this (globals/runtime resolver are not in this picture)

We fixed our issue with the after_context_created, all good now.

the parameters.yml:begin_date is not visible during catalog reading so the interpolation fails

The isolation is intended

Yes got that, it is only the runtime_params:* may be inconsistent. (In our pipelines - and we have been users since 0.16) - the parameters.yml + CLI are part of the catalog filenames (how runs are identified).

kedro run --params

- perhaps should be better name as

kedro run --runtime_params

(the renaming didn't happen mostly for backward compatibility)

filepath: "${globals:some_bucket}/p/q/r/data/date=${runtime_params:begin_date}/_${runtime_params:end_date}.parquet"

Does it not work with

kedro run --params begin_date=xxxx, end_date=xxxx

?

yes, that use case works, fails when parameter.yml is the only source.

As

parameters.yml

itself - is not a runtime parameter. One of the key goal of having runtime_params and globals is to make the resolution path explicit, we try to avoid parameters being reference in arbitrary direction and make it very hard to understand the source of parameters: 1.

parameters

only see

parameters_xxxx.yml

2.

catalog

only see

catalog_xx.yml

If something need to be shared across different configuration group, the standard path is use

$runtime_params

and

$globals

So to do that the change you need to make is: 1. update

parameters

- make it refer to global instead, i.e.

begin_date :${globals:begin_date}

2. in Catalog, then you can have i.e.

some_path/${runtime_params:begin_date, $globals.begin_date}

(take runtime, if not take the global as default)

https://docs.kedro.org/en/stable/configure/advanced_configuration/#how-to-use-globals-and-runtime_params

OK, will try that route - although in our case a bucket is a global concept and the begin/end date are parameters (not globals). I do think the inconsistency remains, where runtime_params may return different result depending where is invoked.

Could you give us an example where the runtime param return different results? It may be the terminology- but you were trying to refer a parameters in catalog and this is against the pattern. Instead you think of it as both your parameters and catalog are referring to the same globals The only special class here is parameter where you can optionally not use the $runtime_params, again this exist only for backward compatibility and should be treated as a special case

Could you give us an example where the runtime param return different results?

As per above, referring to runtime_params in catalog vs during the node/ pipeline execution may differ. This code updates runtime_params with parameters.yml later and if we refer to runtime_params in catalog we get CLI params only.

It may be the terminology- but you were trying to refer a parameters in catalog and this is against the pattern.

Understood. (for us a natural way to think is: globals - values that apply to multiple runs and change rarely (example staging, prod, dev buckets etc); parameters - something that may change per each run - examples are begin/end date, geo, training epochs etc.

As per my understanding -

parameters - something that may change per each run

This we call as runtime parameters that are specific to a run and need to be passed with

kedro run --params=param_key1=value1,param_key2=2.0

As you pointed out, at catalog level we look for CLI params for resolution.

@Emil Marcetta we solved this by using run configurations as explained here You can use a file as follows

run:
  tags: nightly
  pipeline: analysis
  env: prod
  params:
    begin_date: 2025-10-16

and run it using

kedro run --config=<filepath>

The

params

key is cast to the

runtime_params

, which can be interpolated in the catalog This allows you to have it as a configuration file (which can even be created programatically)

Thanks @Arnaud Dhaene - the

--config=<filepath>

is another way of specifying CLI params; we run a number of environments ci, dev, staging prod etc and in some environments (say ci) we simply use values from parameters.yml; we wrote a tiny hook described here that helped our use cases.