Hi, I am a bit confused with the inclusion of optional dependencies (maybe related to this). If I understand the working of optional dependencies in

pyproject.toml

, I should be able to install the group

spark

in

kedro-datasets

with

pip install kedro-datasets[spark]

(I tested the syntax with

pip install httpx[socks]

). But it does not seem to work as

pyspark

is not installed. It works with

pip install kedro-datasets[spark.SparkDataSet]

. Thanks in advance for pointing me to my mistake. :)

hi @Flavien, in kedro-datasets 1.5.0 we messed up the optional dependencies a bit, we're working on a solution that will see the light soon (hopefully tomorrow or Monday)

Excellent! Thanks for the reply and lifting my confusion. 😅

hi @Flavien, kedro-datasets 1.5.3 fixes this issue, please try again!

Let me do that now. 👍

fantastic, thanks for checking!