aiqbench analysis
This systematic review examines how LLM-assisted coding can produce insecure code and faulty patches, and how the models themselves can be attacked directly. It also flags that current benchmarks suffer from data contamination and lack robust security metrics, so the reliability of reported results is uncertain. For AI readers, the practical takeaway is that human review remains necessary, and that security-specific evaluations and model designs are still underdeveloped.
Security in LLM-Assisted Coding: A Systematic Literature Review
arXiv:2610.15482v1 Announce Type: cross Abstract: Large Language Models accelerate software development but introduce severe security vulnerabilities. This systematic review investigates security risks in automated coding, benchmark limitations, and the role of human developers. Findings indicate that models frequently generate insecure code and faulty patches due to training data biases. Additionally, the models themselves are susceptible to direct attacks. Current evaluation methods suffer from data contamination and lack robust security metrics. Consequently, human oversight remains essential. Safer integration of automation will require more security-aware model architectures, stronger security-specific benchmarks, and more effective human-AI collaboration frameworks to mitigate these systemic risks.
Abstract:Large Language Models accelerate software development but introduce severe security vulnerabilities. This systematic review investigates security risks in automated coding, benchmark limitations, and the role of human developers. Findings indicate that models frequently generate insecure code and faulty patches due to training data biases. Additionally, the models themselves are susceptible to direct attacks. Current evaluation methods suffer from data contamination and lack robust security metrics. Consequently, human oversight remains essential. Safer integration of automation will require more security-aware model architectures, stronger security-specific benchmarks, and more effective human-AI collaboration frameworks to mitigate these systemic risks.
35 pages Accepted for publication in ACM Computing Surveys (CSUR). A copy of the formal acceptance letter can be verified via the publisher's portal or provided to moderators upon request
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
arXiv:2610.15482 [cs.CR]
(or arXiv:2610.15482v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2610.15482 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 149 words
Related coverage
Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · Aug 3, 2026
Game-Theoretic Disempowerment: Loss of Control to Agencyless AI
LessWrong · Sep 27, 2026
Small LLMs perform no-CoT two-hop reasoning on synthetic facts with real bridges
LessWrong · Oct 17, 2026
Models Can't Remember Their Training. Neither Can You.
LessWrong · Jul 19, 2026