Does anyone have an example of how to access params passed at the CLI in a hook? I know this isn't a very Kedronic pattern, but it seems a necessary workaround for my use case

I’d argue it is Kedriffic! The params should be in the

extra_params

attr of the

before_pipeline_run

hook

I'll add that one to my vocabulary! Do you have any thoughts about if this will work if I'm trying to dynamically generate catalog entries? I think what I really need is to be able to override globals from the CLI, something like this:

${<http://globals.site|globals.site>}_${globals.service}_usage:
  type: pandas.SQLQueryDataSet
  filepath: data/01_raw/${globals.service}.sql
  load_args:
    index_col: "my_index_col"
    params:
      start_date: ${globals.start_date}
      end_date: ${globals.end_date}
  credentials: ${globals.site.db_creds}
  layer: Raw

But as I understand, this isn't currently possible with catalog templating. Dynamically generating the catalog at runtime via hooks seems like the next best option.

You can do it with a omega conf resolver

Thanks, I'll look into this!

hey let me look into this

I'm looking at this page of the docs: https://docs.kedro.org/en/latest/configuration/advanced_configuration.html#how-to-use-resolvers-in-the-omegaconfigloader\

So if you look here https://kedro-org.slack.com/archives/C03RKP2LW64/p1692205734659679?thread_ts=1692160799.368559&cid=C03RKP2LW64

Oh yeah I saw this, but I wasn't sure where

ExtendedOmegaConfigLoader

was implemented (within

settings.py

?) or what the contents of

register_custom_resolver

was supposed to be!

it can be anywhere you just need to import it into setting.py and use it under

CONFIG_LOADER_CLASS

Cheers, that makes sense! It would be good to know what's going on in that static method 🤔

@Fazil B. Topal are you able to share more of your snippet?

that is the whole snippet so far

@Fazil B. Topal Thank you very much for this!

@Jordan Barlow Did this end up working for you? OmegaConf looks undefined to me in the static method above. @Fazil B. Topal @datajoely any help appreciated. Edit: nvm was being silly. Just had to import OmegaConf from omegaconf. Sorry for the pings.