Hello All! I wanted to implement a credential res...
# questions
n
Hello All! I wanted to implement a credential resolver function similar to this one: https://github.com/kedro-org/kedro/discussions/4320#discussioncomment-12141508 However, I do not want to put the conf_source derivation logic and the catalog read logic in here like @datajoely has put in
KedroCredentialResolver.get_credentials
, as my catalogs can be present anywhere (local system / s3) depending on scenario. Is there any way to access some kind of pre-loaded config or catalog object so that I do not need to look for and read the credentials files myself, and instead I can just rely on the pre-loaded conf object to simply resolve the credentials? I would want my catalog to look just the same as in the comment.
catalog.yml
Copy code
my_dataset:
   type: ... 
   password: "${creds: snowflake_credentials, password}"
and
creds.yml
Copy code
snowflake_credentials:
   password: "<secretsvault://secret-name>"
and my resolver would be able to use the secretvault url provided in
creds.yml
to fetch the secret whenever required inside the dataset. Is there any way to achieve something close to this? The reason for my interest in this implementation is that we have custom ways of getting credentials depending on the different scenarios and we want to make it very easy for someone to refer to them. We are okay with having to use custom implemented datasets to get this to work. However, having the above mentioned implementation combined with custom datasets could probably give the cleanest user-side catalog. Older question along similar lines: https://kedro-org.slack.com/archives/C03RKP2LW64/p1761746217970269
It looks like, OmegaConf allows sending
_root_
into resolvers: https://omegaconf.readthedocs.io/en/latest/custom_resolvers.html So, I can have a custom section in parameters:
Copy code
creds:
  credA:
    url: secret-vault/xyz
and then access this using
_root_
from inside the resolver. Please let me know if there are any other better suggestions for this whole use case.