Hi Everyone, I am trying to load a file from an SFTP server and am facing the following error:

DataSetError: Failed while loading data from data set CSVDataSet(filepath=/home/foo/dev.csv, load_args={}, protocol=sftp, save_args={'index': False}).
<urlopen error unknown url type: sftp>

The file is referenced in

conf\base\catalog.yml

using the following syntax:

input_data:
    type: pandas.CSVDataSet
    filepath: "sftp:///home/foo/dev.csv"
    credentials: cluster_credentials

Where the cluster_credentials are as follows in my

conf\local\credentials.yml

if

cluster_credentials:
  username: username
  host: localhost
  port: 22
  password: password

I am running Kedro version 0.18.8 and I have Paramiko version 3.2.0 installed running on a Windows machine. I have followed the instruction in the data catalog docs here. I would greatly appreciate any insights or suggestions on how to debug and resolve this issue. Thank you in advance for your help! Best Regards Hannes

Potentially related: https://github.com/kedro-org/kedro/discussions/1205

@Nok Lam Chan can you have a look at this. Thanks.

I have resolved the issue by using the following dataset type in the catalog:

input_data:
    type: pandas.GenericDataSet
    filepath: "sftp:///dev.csv"
    file_format: csv
    credentials: cluster_credentials