Hey, we’re running kedro on a Databricks job and w...
# questions
p
Hey, we’re running kedro on a Databricks job and when hitting an exception we see: 1. the log of the job shows HTML-like traceback messages, 2. the exception doesn’t seem to be propagated appropriately to Databricks, since the job shows Status=Succeeded. This seems to be the problem raised in https://github.com/Textualize/rich/issues/2455 and initially fixed with https://github.com/kedro-org/kedro/pull/1769. However, the fix doesn’t seem to be working anymore - could it be that a change in
rich
or Databricks has invalidated the previous approach? We’re using kedro 0.18.3.
d
if you set the
DATABRICKS_RUNTIME_VERSION
to
1
does it work?
or actually
do you have that env var?
does it now say 2 or something?
p
yes, we printed it at the beginning of the job, DATABRICKS_RUNTIME_VERSION=“9.1”
d
so 0.18.3 has a rich dependency of
rich~=0.12.0
whilst 0.13.x came out a couple of weeks ago I don’t see anything in 0.12.x that should have broken this
I don’t see why line 212 in PR would be hit if the env var is present either
p
@datajoely should I open an issue on github?
d
Please