Hi dear kedro community,
I have a question regard...
# questionsp
Patrick H.
01/11/2023, 5:43 PMHi dear kedro community,
I have a question regarding the data catalog and the data itself. So far I can only declare the data filewise in the catalog.yml file. But is it possible to declare let's say... a folder (in my case full with images of same extension)?
It feels out of place to write a function in order to compress the images in one file without putting it in the pipeline, since I must read the folder.
I thank you in advance for the response.
d
datajoely
01/11/2023, 5:44 PMdatajoely
01/11/2023, 5:44 PMPartitionedDataSet and it’s sibling IncrementalDataSet are your friends here!
p
Patrick H.
01/11/2023, 5:44 PMamazing! Thank you so much for the prompt answer
Patrick H.
01/11/2023, 5:45 PMI did not know where to look at it is so much information, sorry for the trivial question
d
datajoely
01/11/2023, 5:45 PMout of interest - what did you search for? We could improve the docs
p
Patrick H.
01/11/2023, 5:45 PMfolder as data dataset
Patrick H.
01/11/2023, 5:45 PM" "
Patrick H.
01/11/2023, 5:47 PMor other iterations with data and folder as keywords
Patrick H.
01/11/2023, 5:48 PMOh wait you're the guy who gave the crash course on yt, thanks so much!
🥳 1
d
datajoely
01/11/2023, 5:54 PMinteresting
datajoely
01/11/2023, 5:54 PMcc @Jo Stichbury wonder if we could change the title of one of the headings to match this search term
j
Jo Stichbury
01/11/2023, 6:03 PM100% and I think that page is in the queue for some refactoring. I'll note this.
❤️ 1
2 Views