Hi everyone i am currently developing a project wh...
# questions
Hi everyone i am currently developing a project which uses GCP credentials. The problem i am facing is that i want to deploy a container of this pipeline but the secrets should no be in the container. I want to know if by using kedro-docker package the secrets are not added to the docker container or in case they do what can i do to handle these credentials with the docker deployment šŸ˜„ thanks
You can check the .dockerignore file that
kedro docker init
creates, I believe it ignores the credentials file. Then you can pass the credentials as environment variables to the container and modify the
so it calls
kedro run
with the creds as parameters. There might be another way but something like that should work
@Jose Alejandro MontaƱa Cortes the best approach would be to rely on Application Default Credentials. Could you share some details on how youā€™re attempting to run this on GCP?
Hi, i am trying to run the pipeline as a job by uploading the docker image. i configure the resources it will use and i just run it. The thing is taht i do not see where can i add the credentials so kedro can understand where it should get them from
i am trying to run the pipeline as a job by uploading
In which GCP service, Vertex AI, AI Pipelines (kubeflow)?
When you donā€™t provide credentials explicitly the default is that Google takes the credentials of service account of a virtual machine thatā€™s running your job.
Could you share the command you are using to run the job?
Hi, what we are doing is in cloud functions we create a job to run a docker image and we set up the resources and we just press the button execute. the image is uploaded to the registry.. So about the commands are just the commands that kedro docker create in the Docker file, basically kedro run
If you are using CloudRun the drill should be the same as for other services, If you donā€™t specify service account to run your cloud run job explicitly, it uses compute engine default service account and that is the jobā€™s identity https://cloud.google.com/compute/docs/access/service-accounts#default_service_account
You shouldnā€™t worry about providing credentials, just grant this service account required accesses. Application Default Credentials logic Iā€™ve mentioned before will take care of authorising the account.
šŸ„³ 1
If you have a specific service account you want to use with your job, then follow this documentation -> https://cloud.google.com/run/docs/configuring/service-accounts
Ok thanks for your answer