Hello, I am trying to connect to adfs using Databricks, I already have a code that is running in Azure VM, I take the code and try to connect to one of the datasets using

with KedroSession.create(project_path=project_path,package_name="package", env="end") as session:
    session.run(node_names=["ds1"
                             ])

and the connection details are like this

ds1:
  type: "${globals:datatypes.csv}"
  filepath: "abfss://<container>@<acount_name>.<http://dfs.core.windows.net/raw_data/ds1.csv.gz|dfs.core.windows.net/raw_data/ds1.csv.gz>"
  fs_args:
    account_name: "accountName"
    sas_token: "sas_token"
  layer: raw_data
  load_args:
    sep: ";"
    escapechar: "\\"
    encoding: "utf-8"
    compression: gzip
    #lineterminator: "\n"
    usecols:

The token is fine, but I am getting this exception DatasetError: Failed while loading data from data set CSVDataset(filepath=, load_args={}, protocol=abfss, save_args={'index': False}). Operation returned an invalid status 'Server failed to authenticate the request. Please refer to the information in the www-authenticate header.' ErrorCode:NoAuthenticationInformation

Hi @Srinivas, as far as I understand, after some googling the issue is in ADLS protocol mismatch. You’re using

abfss://

(ADLS Gen2 via Azure AD authentication) but providing a SAS token, which only works with

abfs://

or

https://

. While

abfss://

(encrypted Azure Data Lake) expects OAuth / AAD token, not SAS.

can you please point me to the code snippet that tries to connect to adfs, so I can replicate that issue

Kedro internally does

fs = fsspec.filesystem("abfss", **fs_args)

So something like this should give you the same error

import fsspec

fs = fsspec.filesystem(
    "abfss",
    account_name="myaccount",
    sas_token="<valid_sas>"
)

<http://fs.ls|fs.ls>("<abfss://container@myaccount.dfs.core.windows.net/>")