> We collect 64,031 ML notebooks containing 92,...
# resources
j
We collect 64,031 ML notebooks containing 92,542 crashes from GitHub and Kaggle, and manually analyze a sample of 746 crashes across various aspects, including exception types and root causes. Our analysis highlights unique root causes related to notebook semantics, including out-of-order execution and previous cell error, that have not been thoroughly covered in earlier research. Furthermore, we categorize crashes as ML bugs or general Python bugs and examine how the crashes are distributed across different stages of the ML pipeline. Our analysis reveals that 87% of crashes are caused by API misuse, data confusion, notebook-specific issues, environment problems, and implementation errors.
https://arxiv.org/abs/2411.16795 (via Greg Wilson)
interesting