I have a question related to `OmegaConf`and custom resolvers. Some context first - client is using https://getindata.com/blog/kedro-dynamic-pipelines/ with custom resolver

merge_dict

which allows to combine multiple keys in the configuration and merge them (deep merge) as in the example below:

model_options:
  test_size: 0.2
  random_state: 3
  target: costs
  features:
    - engines
    - passenger_capacity
    - crew
    - d_check_complete
    - moon_clearance_complete
    - iata_approved
    - company_rating
    - review_scores_rating
  model: sklearn.linear_model.LinearRegression
  model_params: {}

price_predictor:
  _overrides:
    target: price
  model_options: ${merge:${model_options},${._overrides}}

  base:
    model_options: ${..model_options}
  
  candidate1:
    _overrides:
      features:
      - engines
      - passenger_capacity
      - crew
      - d_check_complete
      - company_rating
    model_options: ${merge:${..model_options},${._overrides}}

The problem is when the merge needs to happen between

local

and environment config - it seems like the resolvers are “resolved” ( 😄 ) for each environment (=each file) separately. Question: Is there a way to influence that behaviour and be able to do “config merge” first and then apply resolvers? 🤔

Would this help replace the need for a merge resolver? - https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-change-the-merge-strategy-used-by-omegaconfigloader

I don’t think so - see, the

merge

resolver is mostly do do “inheritance” across keys - as in the example above - you have some “base parameters” that you want to use and then multiple “namespaced” instances of those params - in this case it’s a model (

price_predictor

) with multiple variants (

base

,

candidate1

, etc.). If

local

env wants to overwrite top level

model_options

, they will not be used in the

model_options: ${merge:${model_options},${._overrides}}

I see, I know

globals

are loaded and merged first but even there the config is resolved per environment and then merged. But when you refer to it in

parameters.yml

, the overridden key from the local env is used for the resolution - maybe a combination of

merge

and

globals

resolver?

Could you give a short example?

Yep, let me actually try it out ^

Yup, that might work. Thanks! 🎉

Late to the party, but almost all configuration are contained within the environment, except

globals

, thus the name. Cross-env interpolation was discussed earlier but we decide not to do it because it creates a lot of confusion