Hi team, See this example of a pipeline, single n...
# questions
y
Hi team, See this example of a pipeline, single node, no inputs, one output. Why does it do the last logged operation: loading a dataset back, after the pipeline is completed successfully? P.S. That doesn't cause any issues on my end, I'm just curious.
s
Hi Yury, I believe Kedro is loading
mi_dataset
after the pipeline finishes because it's an output of your pipeline, and the runner is designed to make these final outputs accessible, which involves a
load
operation from the catalog.
โ˜๐Ÿผ 1
๐Ÿ‘ 1
n
The data is already in memory, so itโ€™s not doing much. If you have the output as a persisted dataset, say a pickle one, that log should vanish