analiza aiqbench
This research proposes a formal specification for auditing generative search results by rigorously validating the logical relationship between cited sources and generated answers, rather than relying solely on citation presence. By treating missing evidence as unresolved states within an executable contract framework, the method addresses critical gaps in trustworthiness where standard baselines might incorrectly assume independence. The work defines necessary evaluation protocols for semantic validity in AI-driven information retrieval systems.
Claim-Gated Source-Risk Auditing for Generative Search
arXiv:2609.29145v1 Announce Type: new Abstract: A generative search answer can cite a supported passage yet omit a source relationship that changes its interpretation. We specify a claim-gated audit of the query-source-answer tuple. An omission is resolved only when relationship evidence, answer adoption, materiality, and disclosure are all observed; incomplete evidence remains unresolved rather than being treated as independence. The specification separates this endpoint from citation support and review priority, and binds decisions to versioned evidence spans. A reference checker makes the record contract executable. On an exhaustive synthetic suite, it reproduces all 81 three-state predicate combinations and rejects 192 deliberately malformed records. Common-guard baselines and predicate ablations isolate endpoint logic from missing-evidence handling, while controlled transitions check support separation and evidence removal. These are finite contract-conformance results, not detector accuracy or evidence of improved user outcomes. We define the independent annotation, held-out evaluation, and paired utility tests still required to establish semantic validity and deployment benefit.
Abstract:A generative search answer can cite a supported passage yet omit a source relationship that changes its interpretation. We specify a claim-gated audit of the query-source-answer tuple. An omission is resolved only when relationship evidence, answer adoption, materiality, and disclosure are all observed; incomplete evidence remains unresolved rather than being treated as independence. The specification separates this endpoint from citation support and review priority, and binds decisions to versioned evidence spans. A reference checker makes the record contract executable. On an exhaustive synthetic suite, it reproduces all 81 three-state predicate combinations and rejects 192 deliberately malformed records. Common-guard baselines and predicate ablations isolate endpoint logic from missing-evidence handling, while controlled transitions check support separation and evidence removal. These are finite contract-conformance results, not detector accuracy or evidence of improved user outcomes. We define the independent annotation, held-out evaluation, and paired utility tests still required to establish semantic validity and deployment benefit.
Artificial Intelligence (cs.AI); Computational Complexity (cs.CC); Computational Engineering, Finance, and Science (cs.CE); Emerging Technologies (cs.ET); Information Retrieval (cs.IR)
arXiv:2609.29145 [cs.AI]
(or arXiv:2609.29145v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.29145 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 184 cuvinte
Subiecte conexe

[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · 17 sept. 2026

[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model
Latent Space · 13 oct. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026

Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · 5 oct. 2026