aiqbench analysis
This research presents CTI2TTP, a novel framework for extracting MITRE ATT&CK techniques from cyber threat intelligence reports by combining broad candidate generation with rigorous evidence-based verification. By decomposing reports into atomic behaviors anchored to specific text spans, the method significantly outperforms existing LLM-based and rule-based approaches in balancing high recall and precision without hallucinating unsupported tactics. The authors also release cleaned benchmarks (TRAM-Clean) and a new dataset (CTI2TTP-Bench) to standardize evaluation in this domain.
CTI2TTP: Evidence-Grounded TTP Extraction from Cyber Threat Intelligence via Diverge-then-Converge Verification
arXiv:2605.25836v2 Announce Type: replace-cross Abstract: Extracting MITRE ATT&CK techniques from cyber threat intelligence (CTI) reports is an open-set, multi-label problem requiring both high recall (not missing techniques) and high precision (not hallucinating unsupported ones). Existing rule-based, supervised, and LLM-based methods struggle to achieve both: rule-based and supervised approaches lack generalizability across diverse attack descriptions, while LLM-based approaches that couple candidate generation and validation within a single inference step suffer from limited recall and precision simultaneously. We propose CTI2TTP, which addresses this challenge through a diverge-then-converge design inspired by how human analysts work: first extracting broadly, then verifying rigorously. Reports are first decomposed into atomic behaviors, each anchored by a deterministic span localization stage to a specific evidence window in the source text. Candidate techniques are then proposed broadly for each behavior, and a convergent verification stage retains only candidates supported by both the localized evidence and the authoritative MITRE definition. We contribute two evaluation resources, a cleaned TRAM benchmark (TRAM-Clean) and a new annotated dataset (CTI2TTP-Bench), to address known annotation noise in existing benchmarks and elevate the task to document-level TTP extraction. On TRAM-Clean and CTI2TTP-Bench, CTI2TTP achieves 76.48% and 87.39% macro-F1 respectively, outperforming chain-of-thought prompting with the same backbone by 63.5% and 29.4%.
Abstract:Extracting MITRE ATT&CK techniques from cyber threat intelligence (CTI) reports is an open-set, multi-label problem requiring both high recall (not missing techniques) and high precision (not hallucinating unsupported ones). Existing rule-based, supervised, and LLM-based methods struggle to achieve both: rule-based and supervised approaches lack generalizability across diverse attack descriptions, while LLM-based approaches that couple candidate generation and validation within a single inference step suffer from limited recall and precision simultaneously. We propose CTI2TTP, which addresses this challenge through a diverge-then-converge design inspired by how human analysts work: first extracting broadly, then verifying rigorously. Reports are first decomposed into atomic behaviors, each anchored by a deterministic span localization stage to a specific evidence window in the source text. Candidate techniques are then proposed broadly for each behavior, and a convergent verification stage retains only candidates supported by both the localized evidence and the authoritative MITRE definition. We contribute two evaluation resources, a cleaned TRAM benchmark (TRAM-Clean) and a new annotated dataset (CTI2TTP-Bench), to address known annotation noise in existing benchmarks and elevate the task to document-level TTP extraction. On TRAM-Clean and CTI2TTP-Bench, CTI2TTP achieves 76.48% and 87.39% macro-F1 respectively, outperforming chain-of-thought prompting with the same backbone by 63.5% and 29.4%.
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2605.25836 [cs.CR]
(or arXiv:2605.25836v2 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2605.25836 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 299 words
Related coverage
BEACON: Behavior-Anchored Cross-Source Knowledge Graph Construction for Cyber Threat Intelligence
arXiv · Aug 31, 2026
[AINews] not much happened today
Latent Space · Sep 18, 2026
TRACE-CTI: Auditable Post-Extraction Governance of TTP Claims with Knowledge Graphs
arXiv · Jul 28, 2026
Underwriting Superintelligence: Backing Agents you can Sue — Rune Kvist, AIUC
Latent Space · Sep 16, 2026