#questions
Title
# questions
a
Alexandra Lorenzo
01/31/2023, 4:48 PMHello,
First, thanks a lot for creating such a community.
I'm trying to connect my PartitionedDataSet to my S3 Bucket, I have the following error:
"create_client() got multiple values for keyword argument 'aws_access_key_id'." Copy code
dev_s3:
client_kwargs:
aws_access_key_id: AWS_ACCESS_KEY_ID
aws_secret_access_key: AWS_SECRET_ACCESS_KEY Copy code
raw_images:
type: PartitionedDataSet
dataset:
type: flair_one.extras.datasets.satellite_image.SatelliteImageDataSet
credentials: dev_s3
path: <s3://ignchallenge/train>
filename_suffix: .tif
layer: rawd
datajoely
01/31/2023, 4:49 PMif you look at the example in the docs
the credentials argument needs to be under the
dataset👍 1
as it’s an argument to the wrapped dataset not the partition wrapper
a
Alexandra Lorenzo
01/31/2023, 4:52 PMThanks @datajoely , still the same error
d
datajoely
01/31/2023, 4:53 PMso this one is hard to debug without seeing the implementation from your custom dataset
as it may be how that constructor is configured
👍 1
all we’re doing behind the scenes is importing that class and passing the arguments to it
I find sometimes importing it in a notebook and using the Python API is a good way to debug
👍 1
a
Alexandra Lorenzo
01/31/2023, 4:55 PMWithout context indeed it's not simple.
The code below is working well (without S3), once I tried to read from S3 I have some errors
Copy code
raw_images:
type: PartitionedDataSet
dataset:
type: flair_one.extras.datasets.satellite_image.SatelliteImageDataSet
path: /home/ubuntu/train
filename_suffix: .tif
layer: rawd
datajoely
01/31/2023, 5:03 PMcould you put a breakpoint in your custom dataset and work out what’s not looking correct?
a
Alexandra Lorenzo
01/31/2023, 5:57 PMFound a solution, if I don't specify the credentials parameter everything looks fine, I can access to the S3 buckets.
This error seems corrected in newer version of Kedro.
Thanks @datajoely for your time
d
datajoely
01/31/2023, 5:58 PMawesome!
4 Views