https://kedro.org/ logo
Join Slack
Powered by
I am using the below Docker file created from kedr...
# questions
m
I am using the below Docker file created from kedro-docker plugin and added ENV DB_ENVIRONMENT=${DB_ENVIRONMENT_URL} to it. 
ARG BASE_IMAGE=python:3.9-bullseye
FROM $BASE_IMAGE as runtime-environment
ENV DB_ENVIRONMENT=${DB_ENVIRONMENT_URL}
# install project requirements
COPY src/requirements.txt /tmp/requirements.txt
RUN apt-get update && apt-get install -y libgdal-dev
RUN pip install -r /tmp/requirements.txt && rm -f /tmp/requirements.txt
# add kedro user
ARG KEDRO_UID=999
ARG KEDRO_GID=0
RUN groupadd -f -g ${KEDRO_GID} kedro_group && \
useradd -m -d /home/kedro_docker -s /bin/bash -g ${KEDRO_GID} -u ${KEDRO_UID} kedro_docker
WORKDIR /home/kedro_docker
USER kedro_docker
FROM runtime-environment
# copy the whole project except what is in .dockerignore
ARG KEDRO_UID=999
ARG KEDRO_GID=0
COPY --chown=${KEDRO_UID}:${KEDRO_GID} . .
EXPOSE 8888
CMD ["kedro", "run"]
My catalog loads data from SQL DB and the connection strings are in bash_profile and the credentials.yml file fetches and runs the project in local environment. Here is how it is defined in conf/base/credentials.yml: 
dev_sql:
con: "${DB_ENVIRONMENT_URL}"
When the project is dockerized with above Docker file, and run the docker image it gives the below error: 
<http://kedro.io|kedro.io>.core.DataSetError:
``con` argument cannot be empty. Please provide a SQLAlchemy connection string..` `Failed to instantiate DataSet 'municipalities' of type 
kedro.extras.datasets.pandas.sql_dataset.SQLQueryDataSet
.` It looks like ENV DB_ENVIRONMENT=${DB_ENVIRONMENT_URL} is not getting any value. How is it possible to pass the variables?
d
which config loader are you using?
m
Copy code
class MyTemplatedConfigLoader(TemplatedConfigLoader):
    def __init__(self, conf_source, env, runtime_params):
        self.params = os.environ
        super().__init__(conf_source=conf_source, env=env, runtime_params=runtime_params, globals_dict=self.params)


CONFIG_LOADER_CLASS = MyTemplatedConfigLoader
d
Can you put a breakpoint in your Kedro code and confirm the env var is visible to python?
What you’ve posted looks correct which is weird
m
Breakpoint in kedro code to run it local ? If so, it works when run in Pycharm IDE. And the issue is in docker run
d
I meant inside docker - I guess since you’e confirmed it works locally it looks like the issue is in your docker file which I’m not an expert in
m
Ahh it works if the .env file is passed to docker run 
docker run  --env-file=.env image_name
. So it appears that docker environment does not have the env variables available. Any tip if it is safe to copy env file within the Docker file?
d
I wish I could help, but it’s a bit outside of my area of expertise
glad its working though!
✌️ 1
5 Views
Open in Slack
PreviousNext
Powered by