https://kedro.org/ logo
#questions
Title
# questions
m

Merel

04/20/2023, 3:17 PM
Can anyone with Spark skills lend a hand and help fixing the Kedro pyspark-iris starter?
pyspark
3.4.0 was released on the 13th of April and has broken our
pyspark-iris
. I’ve written up my findings so far in an issue: https://github.com/kedro-org/kedro-starters/issues/123 but it could be I’ve been approaching this all wrong and I’ve now reached the point where I could really use some help figuring out what is going on 🙏
batsignal 1
1
spark 1
m

Matthias Roels

04/20/2023, 7:19 PM
I can have a look next week Monday if you want. I have written lots of pyspark code in the past so who knows…
m

Merel

04/21/2023, 10:49 AM
Thanks Matthias! @Deepyaman Datta found the issue. It was caused by a
None
value being passed to
SparkSession.builder.appName()
👍 1
d

Deepyaman Datta

04/22/2023, 2:22 PM
FYI it's a minor fix for any existing project (https://github.com/kedro-org/kedro/pull/2533/files) and also fixed for the starters (https://github.com/kedro-org/kedro-starters/pull/125/files)
6 Views