hey guys Anyone know how to pass AWS role arn for kedro Ive

Question

hey guys Anyone know how to pass AWS role arn for kedro Ive been hunting down s3fs and other stuff but not much luck I can work with local aws profile but in cloud I have optiona to pass ARN but couldn t see anyway to pass this Any help is much appreciated

datajoely · Answer

maybe worth building a custom dataset using this <https aws sdk pandas readthedocs io en stable api html>

Fazil B. Topal · Answer

We re using parquet dataset directly I just don t know if we can use ARN as is or not supported so that I need to do custom dataset

datajoely · Answer

cam you not provide `role arn` to `fs args`

Fazil B. Topal · Answer

I just did im getting this guy ```AioSession init got an unexpected keyword argument role arn ```

datajoely · Answer

what version of fsspec do you have installed

datajoely · Answer

oh sorry I m being silly

datajoely · Answer

that s arrow not fsspec

Fazil B. Topal · Answer

``` 2023 6 0 ```

Fazil B. Topal · Answer

I feel like there has to be way to pass this guy without me doing boto stuff smile

datajoely · Answer

image png

datajoely · Answer

in the fsspec code they are doing some ARN regex

datajoely · Answer

maybe its just a bucket prefix

Fazil B. Topal · Answer

No idea I think looks like they just exrtract the raw bucket name here don t care about the arn

Fazil B. Topal · Answer

Is there a way to invoke the pyarrow s3fs like you linked

datajoely · Answer

disappointed

Fazil B. Topal · Answer

Do I need to do anything special there

datajoely · Answer

gt Is there a way to invoke the pyarrow s3fs like you linked no I don t think so