Matthias Roels
04/16/2024, 6:56 PM0.18
, I found out that when you save intermediate data, kedro loads the data from file storage again instead of using the dataset already in memory. This wastes precious I/O operation in my pipeline run. Is there a specific reason why it was implemented this way?Iñigo Hidalgo
04/16/2024, 9:00 PMNok Lam Chan
04/16/2024, 9:38 PMNok Lam Chan
04/16/2024, 9:39 PM