Can anyone with Spark skills lend a hand and help fixing the Kedro pyspark-iris starter?

pyspark

3.4.0 was released on the 13th of April and has broken our

pyspark-iris

. I’ve written up my findings so far in an issue: https://github.com/kedro-org/kedro-starters/issues/123 but it could be I’ve been approaching this all wrong and I’ve now reached the point where I could really use some help figuring out what is going on 🙏

I can have a look next week Monday if you want. I have written lots of pyspark code in the past so who knows…

Thanks Matthias! @Deepyaman Datta found the issue. It was caused by a

None

value being passed to

SparkSession.builder.appName()

FYI it's a minor fix for any existing project (https://github.com/kedro-org/kedro/pull/2533/files) and also fixed for the starters (https://github.com/kedro-org/kedro-starters/pull/125/files)