Juan David Patiño Guerra
09/16/2023, 7:00 PMsession.run()
, without the part below it.
with KedroSession.create(env="databricks") as session:
session.run()
context = session.load_context()
catalog = context.catalog
my_dataset =catalog.load("x_train@pandas")
my_dataset.head()
Yet when I try to access the elements inside the catalog through catalog.load("x_train@pandas")
I get the error:
DataSetError: Data for MemoryDataSet has not been saved yet
I am not sure what is the best way to access the in memory elements that were part of the run, and I presume there must be a way, so any help here would be very helpful!
Thanks in advance for your help community :)Deepyaman Datta
09/17/2023, 3:07 AMI am not sure what is the best way to access the in memory elements that were part of the run, and I presume there must be a way, so any help here would be very helpful!There is no way. `MemoryDataset`s are not persisted anywhere; if you want to access them later, use a physical dataset.
Juan David Patiño Guerra
09/17/2023, 10:31 AM