Hi folks, I encountered a problem related to Omegaconf while upgrading our code from kedro 0.18.14 to 0.19.8. I think that this is an issue I raised before but I can't find the thread anymore. We use a custom resolver which reads

CONFIG_LOADER_ARGS = {
    "base_env": "base",
    "default_run_env": "local",
    # "config_patterns": {
    #     "spark" : ["spark*/"],
    #     "parameters": ["parameters*", "parameters*/**", "**/parameters*"],
    # }
    "custom_resolvers": {
        "parse-iso-8601-duration": parse_iso_8601_duration,
        "parse-unix-epoch": parse_unix_epoch,
    },
}

where

def parse_iso_8601_duration(duration: str) -> timedelta:
    """This function aims to convert an ISO-8601 duration string into a timedelta
    object.

    Args:
        duration (str): eg. "1d" = 1 day, "2h" = 2 hours

    Returns:
        timedelta
    """
    return pd.Timedelta(value=duration).to_pytimedelta()

which converts a string to a timedelta. This resolver was working well when doing something like

kedro run --env example --pipeline example --params duration_iso8601=2d

with the parameters reading

example-duration: ${parse-iso-8601-duration:${runtime_params:duration_iso8601,1d}}

With version 0.19.8, omegaconf raises the following issue

omegaconf.errors.UnsupportedValueType: Value 'timedelta' is not a supported primitive type
    full_key: example-duration
    object_type=dict

Note that it works when the default parameter 1d is used. Thanks in advance for your help!

Hi Flavien, thank you for sharing so detailed description! Could you please also share

OmegaConf

versions you use with 0.18.14 and 0.19.8 Looks like the problem is in changed validation at the

OmegaConf

side: https://github.com/omry/omegaconf/blob/6f9656bf091f6cb1ebfe22c926e60e1c9d25ca02/omegaconf/nodes.py#L142 The solution might be setting

allow_objects=True

for

OmegaConf

but I do not think kedro currently supports this API.

In both cases, 2.3.0. If you switch between versions 0.18.14 and 0.19.8 on the project I shared (adapting the

pyproject.toml

), everything being equal (including the version of

omegaconf

), the pipeline fails. All the 0.19.x versions make the pipeline fail. I am sure that I had the issue before, being entirely related to

runtime_params

but Slack does not give me access to the thread.

I see, thank you. In this case, would you mind opening an issue so we can investigate it?

I think https://github.com/kedro-org/kedro/issues/3620 this issue might be the same problem mentioned here.

Thank you, Ankita, looks like so, yes.

Ah, I knew that mentioned it before. 😅

Issue opened; https://github.com/kedro-org/kedro/issues/4126