Amala
11/09/2022, 10:29 PMAllen Ma
11/10/2022, 5:39 AMAlex A
11/18/2022, 4:40 PMon_pipeline_error
, can I re-start the pipeline?Ian Whalen
11/18/2022, 4:49 PMGood question! I’ve never actually done it
Looking at the inputs forit doesn’t look like you’d actually be able to rerun it easilyon_node_error
Forseems like you’d be able to rerun but honestly it doesn’t seem like either node is really meant for that…on_pipeline_error
Inside on pipeline error you could do something like what happens in the main file
But you’d also have to figure out what node errored out somehow…
really not sure sorry 😞
might be easier to parse the log output of the pipeline failure and run thatOverall, I’m not sure that this is really what those hooks are for… its kind of getting into the orchestration world To add some color on the last point, when a pipeline errors you’ll get something like
You can resume the pipeline run from the nearest nodes with persisted inputs by adding the following argument to your previous command:
--from-nodes "..."
Maybe it would be best to just parse that output from the logs and try to rerun from an orchestrator instead? Seems a little hacky but probably more correct
I’m no expert here though