https://kedro.org/ logo
#questions
Title
# questions
a

Ana Paula Rojas

12/07/2023, 8:50 PM
Hi, I'm loading and saving a databricks Delta Table using databricks.ManagedTableDataset. When I load it as a spark dataframe and save it as the same, I don't have any problem. However, when I save it as a pandas dataframe, I'm getting an error while saving the data. Any helps on why is this happening? I'm using python 3.8.10 and kedro version 0.18.14, Databricks runtime 10.4 Solved: Just upgrade the Databricks runtime to runtime 13.x https://stackoverflow.com/questions/75926636/databricks-issue-while-creating-spark-data-frame-from-pandas
d

datajoely

12/07/2023, 9:04 PM
I think you need to do to_pandas() in the node since you’re still working with a spark data frame
a

Ana Paula Rojas

12/07/2023, 9:18 PM
This is what I have in my node, the data loads correctly and loads as a pandas df without needing to add to_pandas, the problem is when saving it
It was solved 🙂 , wrote the solution on the message
d

datajoely

12/08/2023, 9:52 AM
nice!