03/29/2023, 4:18 PM
hey team, question regarding kedro and spark. is there a way to enable spark log4j to output into the log file / terminal output? I have this line in my spark hook:
doing some reading, it seems like I might need a log4j.properties file and specify the file in spark.yml. is this the right path forward?
03/29/2023, 4:59 PM
this feels right
but not 100% sure
let us know if you find it tricky