kedro pipeline in databricks cannot connect to adfs
I am migrating a kedro pipeline to databricks, I am creating a notebook and I am running a node like this
with KedroSession.create(project_path=project_path,package_name="package", env="end") as session:
session.run(node_names=["ds1"
])
My environment is here
ds1:
type: "${globals:datatypes.csv}"
filepath: "abfss://@.
dfs.core.windows.net/raw_data/ds1.csv.gz"
fs_args:
account_name: "accountName"
sas_token: "sas_token"
layer: raw_data...