Hi everyone, I'm trying to load a MemoryDataset in...
# questions
e
Hi everyone, I'm trying to load a MemoryDataset in the same
KedroSession
, but it gives me an error saying that
Data for MemoryDataset has not been saved yet
. Based from my Logs, the data is already saved in my MemoryDataset before loading it using
session.load_context().catalog.load("memory_data_set")
t
Session.load_context().catalog.load("memory_data_set")
will create a new catalog object using your catalog configs, that’s why you didn’t find your dataset in it. If your dataset is a free outputs, you can get it through the session run results
results = session.run(..)
results.get(‘your_dataset_name’)
Otherwise, what are you trying to achieve with this ?
👍 2
i
Also, datasets are "released" when they're no longer an input to a remaining node. For memorydatasets that means that the data is set to None again https://kedro-org.slack.com/archives/C03RKP2LW64/p1700131814151039
👍 2
e
this makes sense now, will try that out thank you