https://kedro.org/ logo
#questions
Title
# questions
p

Patrick H.

01/11/2023, 5:43 PM
Hi dear kedro community, I have a question regarding the data catalog and the data itself. So far I can only declare the data filewise in the catalog.yml file. But is it possible to declare let's say... a folder (in my case full with images of same extension)? It feels out of place to write a function in order to compress the images in one file without putting it in the pipeline, since I must read the folder. I thank you in advance for the response.
PartitionedDataSet and it’s sibling IncrementalDataSet are your friends here!
p

Patrick H.

01/11/2023, 5:44 PM
amazing! Thank you so much for the prompt answer
I did not know where to look at it is so much information, sorry for the trivial question
d

datajoely

01/11/2023, 5:45 PM
out of interest - what did you search for? We could improve the docs
p

Patrick H.

01/11/2023, 5:45 PM
folder as data dataset
" "
or other iterations with data and folder as keywords
Oh wait you're the guy who gave the crash course on yt, thanks so much!
🥳 1
d

datajoely

01/11/2023, 5:54 PM
interesting
cc @Jo Stichbury wonder if we could change the title of one of the headings to match this search term
j

Jo Stichbury

01/11/2023, 6:03 PM
100% and I think that page is in the queue for some refactoring. I'll note this.
❤️ 1
2 Views