Hi, I have a dataset that takes a connection string as config. I want to keep the db auth(user/pass) in credentials.yml and interpolate them into the connection string in the catalog. Is this possible?

Definitely is! You'll likely have to make your own dataset though You could look at the APIDataSet for inspiration You can accept your credentials in as whatever type works for you (tuple, dict, etc) and then format the connection string and use it in your

_load

method when needed

It's already a custom dataset. I was mostly asking if interpolating them into the connection string in the catalog.yml is possible before I spend effort trying to continue debugging why it doesn't seem to work

That sounds right? I think I've run into something similar in the past... It might be easiest to format the credentials into the connection string in your custom dataset So you could leave the base address as in catalog and user/pass in your credentials From the docs: https://docs.kedro.org/en/stable/data/data_catalog.html#dataset-access-credentials Credentials are treated special by the data catalog loading step But maybe someone will come along with a better suggestion! I could be wrong

ty 🙏

It should be possible with some custom omegaconf resolvers, didn't test it though. You might try to force resolving loading credentials config before catalog