https://kedro.org/ logo
#questions
Title
# questions
s

Santiago Zuluaga

02/20/2024, 8:54 PM
Hi, I´m trying to do a automatic retry execution if there´s an error in a node of my pipeline. (I do this basically because a request or the scrapping that I do in my process could fail). I do this with hook
on_node_error
, in the hook I simply do this:
Copy code
class RetryExecution():
    @hook_impl
    def on_node_error(self, node: Node, error, catalog, inputs):
        print(f"The exception was:{error}")
        print("Retrying execution...")
        if error:
            with KedroSession.create() as session:
                session.run()
But this create a second session, and doesn´t retries the first one. Someone knows if there´s a way to do this better? Also, is there a way to retry the execution but from node where it failed. Thanks!
j

Juan Luis

02/21/2024, 7:33 AM
there used to be a decorator in Kedro to retry nodes https://docs.kedro.org/en/0.17.7/_modules/kedro/extras/decorators/retry_node.html could you try this approach? I think it will work better than the
on_node_error
hook