Hey team, can we set a way for the pipeline not to fail if the catalog entry does not exist? Is there a way to set a default None value if the catalog does not return a directory, will this be set in the hooks.py file?
05/17/2023, 10:51 AM
hi @Dotun O, you could indeed try to use the
. not sure which one would be more appropriate. we have a number of issues to improve the documentation around hooks, feedback welcome
05/17/2023, 3:11 PM
I used before_pipeline_run
05/17/2023, 3:14 PM
and did it work?
Nok Lam Chan
05/17/2023, 3:56 PM
Can I ask why do you want this? If the data doesn’t exist the pipeline will be failed anyway