If I have parameters like so ```dataset_config: ...
# questions
If I have parameters like so
Copy code
   primary_key: [date, timeseries_id, unit_id]
Using OmegaConf resolvers could I somehow select
in another parameter?
I'm looking at the built-in resolvers but it's a bit unclear if they serve the purpose I need
I think you need to register something like this
lambda x,y : x[y:]
and then I would do
Copy code
    "custom_resolvers": {
        "index_list": lambda x, y: x[y:]
${index_list:dataset_config.primary_key, 1}
Yes exactly
testing now, ill report back in a couple mins 🙂
🤞 1
got sidetracked as i just discovered relative interpolation 🤯 https://omegaconf.readthedocs.io/en/2.3_branch/usage.html#id26
this is a huge gamechanger for me btw, idk if it's in the documentation specifically, but it makes some config sososo much easier!!!
❤️ 2
it seems i'm missing a small thing on the resolver btw, taking this example
${index_list:dataset_config.primary_key, 1}
this is returning
So it's parsing my interpolation key as a string, This works though:
${index_list:${dataset_config.primary_key}, 1}
so there is no requirement for it to be a Lambda I guess, if you provide a
can you put a breakpoint in it?
yeah, it was just a question of omegaconf not interpreting just the pure string as a thing to be interpolated, but my final line works exactly as it should!! so my question is fully solved. I have to say Omegaconf is a huge upgrade, and it seems I'm only scratching the barest surfacrt
❤️ 1
i have to say relative imports are actually a bigger find for me than the custom resolver
K 1
but it's only really super useful within the scope of dynamic pipelines @marrrcin
👍 1
it’s honestly such a huge huge moment for the team @Merel deserves a lot of credit for the thoughtful design and @Ankita Katiyar for doing most of the implementation (sorry if I forgot anyone) 🚀. The opus of config research written by @Hamza a lonnnnng time ago underpinned most of this thinking too. I have Issue 891 etched into my brain lol. The config system has been the one of the biggest blockers for us doing some of the more ambitious features we have planned since we knew we were going to replace it and it was embedded so deep down it touched EVERYTHING. Now we’ve shipped it OmegConf - not only do you have new super powers like the above, but it unlocks so much we’re now in a position to cook up 👨‍🍳
❤️ 7
I wanna say huge kudos to everybody involved in all the development on kedro over the past couple of years. The difference in UX from 0.17.1 and current kedro is incredible. The basics are the same but it's become such a pleasure to use and develop. Whereas before it felt like sacrifices were made in order to fit the "kedro" way, now it feels like we have the necessary tools to extend the kedro way to fit many different needs.
🥹 3
❤️ 8
Thank you for your kind words @Iñigo Hidalgo, your feedback is incredibly valuable and it makes my day reading a comment like this 🥰
🎉 1
Don't forget to leave a star on https://github.com/omry/omegaconf too :D
🌟 2
it's so exciting to read this thread, especially knowing that it's been a looooong journey for you @Iñigo Hidalgo to migrate these pipelines from ancient Kedro versions!! thanks a bunch for your heartwarming feedback K
Hi @Iñigo Hidalgo Thanks for this information! I am working on something similar at the moment. Just one question, where exactly in the code do you call:
${index_list:${dataset_config.primary_key}, 1}
? Do you then have
primary_key: [timeseries_id, unit_id]
available in a dict?
Hi Janick It would be a further parameter, so
Copy code
  primary_key: [date, timeseries_id, unit_id]


    single_timeseries_id: ${index_list:${dataset_config.primary_key}, 1}
I could then access
which would be passed to the node as
[timeseries_id, unit_id]
migrate these pipelines from ancient Kedro versions
@Juan Luis i have to admit, i cheated there. in the end, the cost-benefit just wasn't in favor of actually migrating that older code to newer kedro. instead i was able to get buy-in to start reimplementing it in a much more modular way.
👍🏼 1