https://kedro.org/ logo
#questions
Title
# questions
m

Merel

07/13/2023, 4:09 PM
Is it possible to directly load an
.xlsx
file into a Kedro
SparkDataSet
?
d

datajoely

07/13/2023, 4:14 PM
no pandas then transcode
you may be able to add some Jar to spark to make it work, but it’s way easier to do this way
esp when an Excel file has to be < 1M lines
m

Merel

07/13/2023, 4:18 PM
Okay cool that’s what I’m doing now 👍
d

Deepyaman Datta

07/13/2023, 11:20 PM
you may be able to add some Jar to spark to make it work, but it’s way easier to do this way
Yeah, plus IIRC performance reading with the Spark Excel connector isn't good anyway, so what's the point