kedro spark session configuration values not found
I am unable to access the kedro spark session configuration from an ipython console.
# /conf/base/spark.yml
spark.driver.maxResultSize: 30g
spark.scheduler.mode: FAIR
spark.driver.memory: 15g
spark.executor.memory: 15g
spark.executor.cores: 4
# settings for the UI
spark.ui.port: 10
spark.ui.enabled: true
As you can see the spark.driver.maxResultSize is defined. after running this I get a NoSuchElementException. Any idea why?
kedro ipython
%reload_kedro
spark =...