[Azure Blob Storage] Hello everyone! I'm having trouble connecting to Azure Blob Storage with kedro. The exception isn't very helpful:

DatasetError: Failed while loading data from data set PickleDataSet

I registered my model as follows in the

catalog.yaml

file:

bulldozers_df_models:
  type: pickle.PickleDataSet
  filepath: <https://kedrodatastores.blob.core.windows.net/my-container/06_models/bulldozers/df.pkl>
  credentials: azure_data_storage
  backend: pickle

In the

credentials.yaml

file I used the following fields:

azure_data_storage:
  DefaultEndpointsProtocol: https
  AccountName: kedrodatastores
  AccountKey: mykey
  EndpointSuffix: <http://core.windows.net|core.windows.net>

Does anyone know what I did wrong?

so the trick to debug this is to go into notebook, import

from kedro.datasets.pickle import PickleDataSet

and get this working. The YAML api is just passing arguments to this constructor - so you can debug it nicely that way

Why do you use

https://

? For Azure Blob Storage you should use

abfs://

the keys of your dict may have a typo. you should check the constructor https://github.com/fsspec/adlfs/blob/01e91b150b7d4e6793d882b0e700b8966b181f95/adlfs/spec.py#L120C1-L120C44 since kedro is just using

fsspec.filesystem

Using abfs and fixing my dictionary keys worked for me. I also needed to install the

adlfs

library. Thank you very much!