I just want to request the feature to inject crede...
# random
b
I just want to request the feature to inject credentials via "credentials:username" into nodes like parameters and datasets - while the current config_loader setup works, it just feels clunky and weird in my opinion, since you always need to get the projects root config path. Is there any good reason why it would be a bad idea to inject the credentials like that?
d
I know this is not always the case, but usually Kedro nodes are more pure data manipulation functions and shouldn't need credentials. If I needed credentials, I think it might make more sense to load a custom "resource" dataset that accepts credentials, and gives a handle to said authenticated resource (rather than working with credentials in the node).
b
Hm, so let's say I want a pipeline which downloads some datasets - so if I understand you correctly, instead of implementing this as pipeline, you would create a custom kedro dataset, accept credentials in init, and then download the dataset in there?
d
If you're downloading datasets, ideally, yes--datasets are the "right" place for data I/O, not nodes. But, let's say you can't do that for whatever reason, what I'm suggesting is that you create a custom
DatasetDownloaderDataset
that accepts credentials and returns an authenticated
DatasetDownloader
object that you can use in your node.
If you are familiar with Dagster, this is kind of like having a dependency on a "resource", rather than a data "asset": https://docs.dagster.io/dagster-basics-tutorial/resources
b
Alright, thank you so much for your reply!