As counterpoint experimental pyspark support

Question

As counterpoint < Nok Lam Chan> experimental pyspark support for duckdb is bonkers Basically replace your connection string

marrrcin · Answer

```from duckdb experimental spark sql import SparkSession as session from duckdb experimental spark sql functions import lit col``` This one is sick open mouth

datajoely · Answer

yeah it s actually really interesting

datajoely · Answer

basically the way Snowpark was made but this one isn t even pretending

datajoely · Answer

image png

datajoely · Answer

it s experimental

Nok Lam Chan · Answer

The DataFrame protocol seems to solve these problem more fundamentally There has been so many library trying to copy pandas Spark API

datajoely · Answer

or ibis