Hi everyone :wave: I'm currently working with Kedr...
# questions
r
Hi everyone šŸ‘‹ I'm currently working with Kedro and trying to load a CSV file hosted on an SFTP server using a
CSVDataset
. Here's the relevant entry from my `catalog.yml`:
Copy code
yaml
Copy code
cool_dataset:
  type: pandas.CSVDataSet
  filepath: 
<sftp://my-sftp-server/outbox/DW_Extracts/my_file.csv>
  load_args: {}
  save_args:
    index: False
When I run:
Copy code
python
df = catalog.load("cool_dataset")
I get the following error: It seems like Kedro/Pandas is trying to use ur`llib` to open the SFTP URL, which doesn't support the
sftp://
protocol natively. Has anyone successfully used Kedro to load files from SFTP? If so, could you share your config/setup?
d
Yes I think point 2 is correct, I’m pretty sure there is an example in the docs
r
I use this example to load my file but it does not work, i get this error :
any help plz ?
j
unfortunately this is a pandas issue we cannot easily solve https://github.com/kedro-org/kedro-plugins/issues/562
@Rachid Cherqaoui here you can find a possible workaround https://github.com/kedro-org/kedro-plugins/issues/562#issuecomment-1959443069
r
it help a lot, thank u Juanlu ^^
j
you're welcome! šŸ™ŒšŸ¼
thankyou 1
ā¤ļø 1