If I have parameters like so

dataset_config:
   primary_key: [date, timeseries_id, unit_id]

Using OmegaConf resolvers could I somehow select

dataset_config.primary_key[1:]

in another parameter?

I think you need to register something like this

lambda x,y : x[y:]

and then I would do

CONFIG_LOADER_ARGS = {
    "custom_resolvers": {
        "index_list": lambda x, y: x[y:]
}}

${index_list:dataset_config.primary_key, 1}

?

Yes exactly

testing now, ill report back in a couple mins 🙂

so there is no requirement for it to be a Lambda I guess, if you provide a

def

can you put a breakpoint in it?

yeah, it was just a question of omegaconf not interpreting just the pure string as a thing to be interpolated, but my final line works exactly as it should!! so my question is fully solved. I have to say Omegaconf is a huge upgrade, and it seems I'm only scratching the barest surfacrt

it’s honestly such a huge huge moment for the team @Merel deserves a lot of credit for the thoughtful design and @Ankita Katiyar for doing most of the implementation (sorry if I forgot anyone) 🚀. The opus of config research written by @Hamza a lonnnnng time ago underpinned most of this thinking too. I have Issue 891 etched into my brain lol. The config system has been the one of the biggest blockers for us doing some of the more ambitious features we have planned since we knew we were going to replace it and it was embedded so deep down it touched EVERYTHING. Now we’ve shipped it OmegConf - not only do you have new super powers like the above, but it unlocks so much we’re now in a position to cook up 👨‍🍳

I wanna say huge kudos to everybody involved in all the development on kedro over the past couple of years. The difference in UX from 0.17.1 and current kedro is incredible. The basics are the same but it's become such a pleasure to use and develop. Whereas before it felt like sacrifices were made in order to fit the "kedro" way, now it feels like we have the necessary tools to extend the kedro way to fit many different needs.

Thank you for your kind words @Iñigo Hidalgo, your feedback is incredibly valuable and it makes my day reading a comment like this 🥰

Don't forget to leave a star on https://github.com/omry/omegaconf too :D

it's so exciting to read this thread, especially knowing that it's been a looooong journey for you @Iñigo Hidalgo to migrate these pipelines from ancient Kedro versions!! thanks a bunch for your heartwarming feedback K

Hi @Iñigo Hidalgo Thanks for this information! I am working on something similar at the moment. Just one question, where exactly in the code do you call:

${index_list:${dataset_config.primary_key}, 1}

? Do you then have

primary_key: [timeseries_id, unit_id]

available in a dict?

Hi Janick It would be a further parameter, so

dataset_config:
  primary_key: [date, timeseries_id, unit_id]

...

somewhere:
  else:
    single_timeseries_id: ${index_list:${dataset_config.primary_key}, 1}

I could then access

params:somewhere.else.single_timeseries_id

which would be passed to the node as

[timeseries_id, unit_id]