#catalog #azureblobstorage Hello Kedro Community, ...
# questions
m
#catalog #azureblobstorage Hello Kedro Community, I try to connect to a csv file in azure blobstorage with help of this example: https://docs.kedro.org/en/stable/data/data_catalog_yaml_examples.html#load-a-model-saved-as-a-pickle-from-azure-blob-storage I adjusted the catalog and created a credentials.,yml with account_name and account_key. When trying to load the csv to a dataframe in a jupyter notebook I get the following error:
Copy code
dict() got multiple values for keyword argument 'account_name'
Any body can help out on what I am missing? Thx in advance!
d
is it possible you’re providing
account_name
via env variables too?
n
Copy code
dev_abs:
  account_name: accountname
  account_key: key
Try to update the example to
Copy code
dev_abs:
  key: accountname
  secret: key
I didn’t check the documentaiton but fsspec at some points changing the keywords (I experienced with s3 before), not sure is it the same case.
m
hm, I just followed the setup in the spaceflight tutorial and want to test a connection to azure blobstorage. I was not able to find the solution yet. using key and secret instead of account_name or account_key does not work in this case.
m
no, i get the following error:
d
could you run
pip list -V
and tell us the version of
Kedro
,
Pandas
,
fsspec
and
adlfs
m
adlfs 2023.10.0 kedro 0.18.14 kedro-datasets 1.7.1 pandas 2.1.1 fsspec 2023.10.0
d
Okay they look right - can I ask you to do something. Open a notebook in the same Kedro environment. I’m
from kedro.datasets.pandas import CSVDataSet
and try to configure
CSVDataSet
live in python, all we’re doing behind the scenes is passing arguments to this class so if you get the same error plugging in the credentials manually we can isolate the issue
m
the import already gives an error:
do I miss somehting?
d
sorry try this?
from kedro.extras.datasets.pandas import CSVDataSet
m
seems to work
d
can you do a
df.load()
m
okay, same error as before.
d
Okay can you try passing the credentials to the
fs_args
argument?
m
same dict error
d
Okay I’m going to escalate this
👍 1
really not sure what’s going on
n
Can you shared what config did you pass?
Copy code
from kedro_datasets.pandas import CSVDataset
import pandas as pd

data = pd.DataFrame({"col1": [1, 2], "col2": [4, 5], "col3": [5, 6]})
cred = {"account_name":"123","account_key":"123"}
dataset = CSVDataset(filepath="<abfs://model/test.csv>", credentials=cred)

dataset.save(data)
I try this and I get authentication error not dict error
d
does it work if you do
.load()
n
slightly awkward error but still autentication related
m
thanks for the escalation @datajoely
i
oh i think i know what the problem is you only need to list container onwards
not the storage account url
so filepath="abfs://raw/blabla.csv"
👍 1
m
the filepath listing container onwards seem to have solved the dict error but now I get a weird authentication error too
i
Can you share some more info? Traceback and (anonymized if necessary, but preserving the structure) how you're authenticating?