aiqbench analysis
This research addresses the statistical fragility of large language models in network intrusion detection by introducing traffic-aware calibration and attack-orbit invariance. The authors prove that excluding attacker-controllable features yields exact coverage guarantees against adaptive adversaries, though this robustness comes with a measurable accuracy trade-off. The work advances AI safety in cybersecurity by providing rigorous mathematical bounds for model reliability under perturbation.
Robust Conformal Intrusion Detection via Traffic-Aware Calibration and Attack-Orbit Invariance
arXiv:2609.19241v1 Announce Type: cross Abstract: Large language models fine-tuned for network intrusion detection emit single-point predictions without statistical validity guarantees. Conformal prediction supplies a finite-sample coverage guarantee, but a threshold calibrated on clean traffic fails once an adversary perturbs controllable network features. We demonstrate this failure across three intrusion detection benchmarks and propose traffic-aware conformal prediction, which calibrates on traffic drawn from the perturbation mechanism an attacker is expected to use and provably restores coverage whenever that mechanism is known and can be sampled. A stronger, adaptive attacker that queries the target model's own score can still degrade this matched-calibration guarantee. We address this second threat model by excluding attacker-controllable features and their deterministic descendants from the scored representation, and prove that this yields an exact, pathwise coverage guarantee rather than a probabilistic bound. Across three independently fine-tuned language model architectures, this representation remains completely unchanged under every evaluated attack attempt, at a quantified seven-to-fourteen-point cost in clean accuracy relative to the unrestricted feature set.
Abstract:Large language models fine-tuned for network intrusion detection emit single-point predictions without statistical validity guarantees. Conformal prediction supplies a finite-sample coverage guarantee, but a threshold calibrated on clean traffic fails once an adversary perturbs controllable network features. We demonstrate this failure across three intrusion detection benchmarks and propose traffic-aware conformal prediction, which calibrates on traffic drawn from the perturbation mechanism an attacker is expected to use and provably restores coverage whenever that mechanism is known and can be sampled. A stronger, adaptive attacker that queries the target model's own score can still degrade this matched-calibration guarantee. We address this second threat model by excluding attacker-controllable features and their deterministic descendants from the scored representation, and prove that this yields an exact, pathwise coverage guarantee rather than a probabilistic bound. Across three independently fine-tuned language model architectures, this representation remains completely unchanged under every evaluated attack attempt, at a quantified seven-to-fourteen-point cost in clean accuracy relative to the unrestricted feature set.
12 pages, 1 figure, 5 tables. Submitted to IEEE Transactions on Dependable and Secure Computing
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
arXiv:2609.19241 [cs.CR]
(or arXiv:2609.19241v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2609.19241 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 273 words
Related coverage
Privacy, Robustness, and Fairness Trade-offs in Federated Intrusion Detection: Geometric Indistinguishability at the Aggregation Interface
arXiv · Sep 5, 2026
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
METR · Aug 26, 2026
Digital Twin-Based Intrusion Detection for Vehicle Powertrain CAN Bus Systems
arXiv · Aug 20, 2026
Cognitive Graph Intelligence for Adaptive and Robust DDoS Attack Detection in Next Generation Networks
arXiv · Aug 19, 2026