https://kedro.org/ logo
#questions
Title
# questions
h

Hannes

06/08/2023, 1:53 PM
Hi Everyone, I am trying to load a file from an SFTP server and am facing the following error:
Copy code
DataSetError: Failed while loading data from data set CSVDataSet(filepath=/home/foo/dev.csv, load_args={}, protocol=sftp, save_args={'index': False}).
<urlopen error unknown url type: sftp>
The file is referenced in
conf\base\catalog.yml
using the following syntax:
Copy code
input_data:
    type: pandas.CSVDataSet
    filepath: "sftp:///home/foo/dev.csv"
    credentials: cluster_credentials
Where the cluster_credentials are as follows in my
conf\local\credentials.yml
if
Copy code
cluster_credentials:
  username: username
  host: localhost
  port: 22
  password: password
I am running Kedro version 0.18.8 and I have Paramiko version 3.2.0 installed running on a Windows machine. I have followed the instruction in the data catalog docs here. I would greatly appreciate any insights or suggestions on how to debug and resolve this issue. Thank you in advance for your help! Best Regards Hannes
i

Ian Whalen

06/08/2023, 2:00 PM
n

Nero Okwa

06/08/2023, 2:18 PM
@Nok Lam Chan can you have a look at this. Thanks.
h

Hannes

06/09/2023, 7:02 AM
I have resolved the issue by using the following dataset type in the catalog:
Copy code
input_data:
    type: pandas.GenericDataSet
    filepath: "sftp:///dev.csv"
    file_format: csv
    credentials: cluster_credentials
K 2