Hi everyone! I am adding a data fabrication step to a project I'm working on. I've created a node which runs the fabricator I'm using outputting a dictionary of inter-related pandas DataFrames. The rest of the pipeline however is built to use spark - I'm struggling to figure out the best way to convert the fabricated DataFrames spark DataFrames. Any suggestions would be greatly appreciated!

Hi Daniel, is this what you need? https://kedro.readthedocs.io/en/stable/tools_integration/pyspark.html