Hey team, can we set a way for the pipeline not to fail if the catalog entry does not exist? Is there a way to set a default None value if the catalog does not return a directory, will this be set in the hooks.py file?

hi @Dotun O, you could indeed try to use the

after_catalog_created

,

before_node_run

,

before_pipeline_run

, or

before_dataset_loaded

. not sure which one would be more appropriate. we have a number of issues to improve the documentation around hooks, feedback welcome

I used before_pipeline_run

and did it work?

Can I ask why do you want this? If the data doesn’t exist the pipeline will be failed anyway