meharji arumilli
08/18/2023, 10:21 AMARG BASE_IMAGE=python:3.9-bullseye
FROM $BASE_IMAGE as runtime-environment
ENV DB_ENVIRONMENT=${DB_ENVIRONMENT_URL}
# install project requirements
COPY src/requirements.txt /tmp/requirements.txt
RUN apt-get update && apt-get install -y libgdal-dev
RUN pip install -r /tmp/requirements.txt && rm -f /tmp/requirements.txt
# add kedro user
ARG KEDRO_UID=999
ARG KEDRO_GID=0
RUN groupadd -f -g ${KEDRO_GID} kedro_group && \
useradd -m -d /home/kedro_docker -s /bin/bash -g ${KEDRO_GID} -u ${KEDRO_UID} kedro_docker
WORKDIR /home/kedro_docker
USER kedro_docker
FROM runtime-environment
# copy the whole project except what is in .dockerignore
ARG KEDRO_UID=999
ARG KEDRO_GID=0
COPY --chown=${KEDRO_UID}:${KEDRO_GID} . .
EXPOSE 8888
CMD ["kedro", "run"]
My catalog loads data from SQL DB and the connection strings are in bash_profile and the credentials.yml file fetches
and runs the project in local environment. Here is how it is defined in conf/base/credentials.yml:
dev_sql:
con: "${DB_ENVIRONMENT_URL}"
When the project is dockerized with above Docker file, and run the docker image it gives the below error:
<http://kedro.io|kedro.io>.core.DataSetError:
``con` argument cannot be empty. Please provide a SQLAlchemy connection string..`
`Failed to instantiate DataSet 'municipalities' of type kedro.extras.datasets.pandas.sql_dataset.SQLQueryDataSet
.`
It looks like ENV DB_ENVIRONMENT=${DB_ENVIRONMENT_URL} is not getting any value. How is it possible to
pass the variables?datajoely
08/18/2023, 11:07 AMmeharji arumilli
08/18/2023, 11:08 AMclass MyTemplatedConfigLoader(TemplatedConfigLoader):
def __init__(self, conf_source, env, runtime_params):
self.params = os.environ
super().__init__(conf_source=conf_source, env=env, runtime_params=runtime_params, globals_dict=self.params)
CONFIG_LOADER_CLASS = MyTemplatedConfigLoader
datajoely
08/18/2023, 11:28 AMmeharji arumilli
08/18/2023, 11:30 AMdatajoely
08/18/2023, 11:32 AMmeharji arumilli
08/18/2023, 11:41 AMdocker run --env-file=.env image_name
. So it appears that docker environment does not have the env variables available. Any tip if it is safe to copy env file within the Docker file?datajoely
08/18/2023, 11:41 AM