https://kedro.org/ logo
#questions
Title
# questions
m

Massinissa Saïdi

01/27/2023, 11:42 AM
Hello kedroids! I have an error that I can't understand:
Copy code
DataSetError: 
botocore.session.session.create_client() got multiple values for keyword 
argument 'aws_access_key_id'.
DataSet 'dataset' must only contain valid arguments for the 
constructor of 'kedro.extras.datasets.pandas.csv_dataset.CSVDataSet'.
I run my code from a
docker-compose
with only one container (for now), I write files in s3. I specified the credentials this way:
Copy code
aws_credentials:
    aws_access_key_id: XXXXXXX
    aws_secret_access_key: XXXXXXX
and my dataframe in
catalog.yml
this way:
Copy code
dataset:
  type: pandas.CSVDataSet
  filepath: ${s3.path}/data/dataset.csv
  credentials: aws_credentials
docker-compose.yml
Copy code
version: '3.7'

services:
kedro:
      build:
        context: .
        args:
            PIP_USERNAME: ${PIP_USERNAME}
            PIP_PASSWORD: ${PIP_PASSWORD}
            PIP_REPO: ${PIP_REPO}
        dockerfile: dockerfile.kedro
        cache_from:
          - ia-churn
      image: ia-churn
      command: kedro run --env prod --pipeline data-processing
      volumes:
        - .:/usr/src/app/
        - ./data/01_raw/:/usr/src/app/data/01_raw
In
conda
environement evrything works. Someone has an idea please ? More informations: I used kedro v0.18.4 and python 3.10
10 Views