I’m making a kedro pipeline that will use the OpenAI API and I want to store my key and base_api in the

credentials

file. What the most kedronic way to do this kind of operation for a pipeline:

import openai

openai.api_key = ...
openai.base_api = ...

Probably a hook right?

do you have something like an Openai Dataset? If not using environment variable

i’m expecting people using my pipeline to change the key a lot so i’d love for it to live in the credentials file i haven’t started implementing anything yet but an openai dataset is probably what i want maybe i should just make them use an env… hm thanks anyway!

Credentials is only accessed by catalog, so you will need a dataset. Alternatively, you could have a hook to inject credentials to os.env so you don't have to set openai.api_key, I assume something like OPENAI_API_KEY exists

yeah it does for sure people seem to use dotenv in the llm community a lot too i’ll figure something out! thanks again nok

Why not

after_context_created

and then

context.config_loader["credentials"]

@Nok Lam Chan?

You mean setting openai.api_key? It will still requires a hook. I don't have strong preference but if the library support an environment variable I prefer to use it. Both should work.