Hello there! Would it be possible to make the

_suggest_resume_scenario

for the resume scenario optional? I’ve been experiencing consistent failures in my pipelines, and while I’m attempting to gather errors with the hook on_pipeline_error, I’ve noticed that this function tends to clutter my log due to its frequent failures in suggesting something. So, the final expection is always due to

_suggest_resume_scenario

and not about the most relevant thing: the actual error.

May make sense on this to share some examples (a small reproducible example would be especially great) in a GitHub issue?

Sure, I would do it. Meanwhile (since Im in a rush 🔥) I will overwrite the

sequential_runner

commenting this part to raise the original expection and not the one originated in

self._suggest_resume_scenario(pipeline, done_nodes, catalog)

for exec_index, node in enumerate(nodes):
            try:
                run_node(node, catalog, hook_manager, self._is_async, session_id)
                done_nodes.add(node)
            except Exception:
                #self._suggest_resume_scenario(pipeline, done_nodes, catalog)
                raise

Do you mean the suggest function itself is causing exception? If so it is a bug and it would be great if you can provide an example that we can look at. On the other hand just thinking what could possibly cause this, do you have hooks that mutate catalog or using dataset factory?