https://kedro.org/ logo
#questions
Title
# questions
i

Ian Whalen

10/11/2023, 1:42 PM
I’m making a kedro pipeline that will use the OpenAI API and I want to store my key and base_api in the
credentials
file. What the most kedronic way to do this kind of operation for a pipeline:
Copy code
import openai

openai.api_key = ...
openai.base_api = ...
Probably a hook right?
n

Nok Lam Chan

10/11/2023, 1:43 PM
do you have something like an Openai Dataset? If not using environment variable
i

Ian Whalen

10/11/2023, 1:45 PM
i’m expecting people using my pipeline to change the key a lot so i’d love for it to live in the credentials file i haven’t started implementing anything yet but an openai dataset is probably what i want maybe i should just make them use an env… hm thanks anyway!
n

Nok Lam Chan

10/11/2023, 1:48 PM
Credentials is only accessed by catalog, so you will need a dataset. Alternatively, you could have a hook to inject credentials to os.env so you don't have to set openai.api_key, I assume something like OPENAI_API_KEY exists
i

Ian Whalen

10/11/2023, 1:49 PM
yeah it does for sure people seem to use dotenv in the llm community a lot too i’ll figure something out! thanks again nok
m

marrrcin

10/11/2023, 1:53 PM
Why not
after_context_created
and then
context.config_loader["credentials"]
@Nok Lam Chan?
👍 1
n

Nok Lam Chan

10/11/2023, 1:55 PM
You mean setting openai.api_key? It will still requires a hook. I don't have strong preference but if the library support an environment variable I prefer to use it. Both should work.
👍 1