Hi, I'm loading and saving a databricks Delta Table using databricks.ManagedTableDataset. When I load it as a spark dataframe and save it as the same, I don't have any problem. However, when I save it as a pandas dataframe, I'm getting an error while saving the data. Any helps on why is this happening? I'm using python 3.8.10 and kedro version 0.18.14, Databricks runtime 10.4 Solved: Just upgrade the Databricks runtime to runtime 13.x https://stackoverflow.com/questions/75926636/databricks-issue-while-creating-spark-data-frame-from-pandas

I think you need to do to_pandas() in the node since you’re still working with a spark data frame

This is what I have in my node, the data loads correctly and loads as a pandas df without needing to add to_pandas, the problem is when saving it

nice!