https://kedro.org/ logo
#questions
Title
# questions
d

Dharmesh Soni

03/14/2023, 5:25 PM
Hi everyone! There are zip files having data in text files stored on the cloud. Is there any native Kedro or PySpark solution to read these zip and eventually text files? Structure of zip files:
Copy code
├── main_folder.zip
│   ├── folder1
│   │   └── text_file.txt
│   └── text_file.txt
d

datajoely

03/14/2023, 5:39 PM
I think the
text.TextDataSet
will read zip files as fsspec supports that
👍 1
that being said if you have an unpredictable directory structure you may need to define your own custom dataset
d

Dharmesh Soni

03/14/2023, 5:57 PM
@datajoely Thank you!
2 Views