Hi dear kedro community I have a question regarding the data

Question

Hi dear kedro community I have a question regarding the data catalog and the data itself So far I can only declare the data filewise in the catalog yml file But is it possible to declare let s say a folder in my case full with images of same extension It feels out of place to write a function in order to compress the images in one file without putting it in the pipeline since I must read the folder I thank you in advance for the response

datajoely · Answer

datajoely · Answer

PartitionedDataSet and it s sibling IncrementalDataSet are your friends here

Patrick H. · Answer

amazing Thank you so much for the prompt answer

Patrick H. · Answer

I did not know where to look at it is so much information sorry for the trivial question

datajoely · Answer

out of interest what did you search for We could improve the docs

Patrick H. · Answer

folder as data dataset

Patrick H. · Answer

or other iterations with data and folder as keywords

Patrick H. · Answer

Oh wait you re the guy who gave the crash course on yt thanks so much

datajoely · Answer

interesting

datajoely · Answer

cc < Jo Stichbury> wonder if we could change the title of one of the headings to match this search term

Jo Stichbury · Answer

100 and I think that page is in the queue for some refactoring I ll note this