aiqbench analysis
This research introduces a data-aware static analysis method that shifts semantic fault detection in machine learning code from post-training validation to the development phase. By integrating data and control flow analysis with API contracts, the approach identifies issues like improper data scaling before model training, thereby reducing computational waste and improving code reliability. The technique demonstrates efficacy on real-world notebooks, offering a practical tool for developers to catch data-specific bugs early in the software engineering lifecycle.
Data-aware Static Analysis: Improving Detection of Semantic Faults in Machine Learning Code Using Data Characteristics
arXiv:2606.09957v1 Announce Type: cross Abstract: Semantic faults specific to the use of machine learning models are a common problem for machine learning developers, causing suboptimal predictions, high computational cost, or incorrect outputs. For example, one may erroneously use unscaled data to train a scale-sensitive model. Machine learning developers detect these faults after training their models and manually analyzing the results, making it an inefficient process. We propose a novel data-aware static analysis approach to detect semantic faults in machine learning code, allowing developers to reveal these bugs while writing code instead of after training the model. Our approach uses combined data and control flow analysis, and API contracts, enabling data-aware reasoning about machine learning code at a high level of abstraction. We highlight the potential of our solution by analyzing a sample of real-world machine learning notebooks, finding that we can detect faults that require a data-aware approach.
Abstract:Semantic faults specific to the use of machine learning models are a common problem for machine learning developers, causing suboptimal predictions, high computational cost, or incorrect outputs. For example, one may erroneously use unscaled data to train a scale-sensitive model. Machine learning developers detect these faults after training their models and manually analyzing the results, making it an inefficient process. We propose a novel data-aware static analysis approach to detect semantic faults in machine learning code, allowing developers to reveal these bugs while writing code instead of after training the model. Our approach uses combined data and control flow analysis, and API contracts, enabling data-aware reasoning about machine learning code at a high level of abstraction. We highlight the potential of our solution by analyzing a sample of real-world machine learning notebooks, finding that we can detect faults that require a data-aware approach.
Software Engineering (cs.SE); Machine Learning (cs.LG)
D.2.2; D.2.4; D.2.5; I.2.6
arXiv:2606.09957 [cs.SE]
(or arXiv:2606.09957v1 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2606.09957 Focus to learn more arXiv-issued DOI via DataCite
https://doi.org/10.1145/3786582.3786805 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 252 words