Hi Team A qq, i am running a kedro pyspark pipeline in hadoop using yarn I am getting an error as follows:

Exception in thread "main" org.apache.spark.SparkException: When running with master 'yarn' either HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.