aiqbench analysis
This study exposes a critical failure mode in AI-driven data pipelines where redundant verification paths, even when using different technologies like Python and SPARQL, can produce identical errors due to shared misinterpretations of source vocabulary. The research demonstrates that relying solely on computational redundancy is insufficient for ensuring data integrity when the underlying semantic definitions are flawed, as seen in this satellite catalogue analysis where multiple independent checks agreed on incorrect counts. For readers focused on reliable AI systems, the key takeaway is the necessity of grounding verification logic directly in source documentation rather than trusting automated consensus among correlated failure modes.
Independent Verification Paths Are Not Independent: A Case Study of Common-Mode Failure in a Satellite Catalogue Pipeline
arXiv:2609.37603v1 Announce Type: cross Abstract: A common safeguard for a data pipeline is redundant computation: derive each published number by two routes built on different technology and refuse to exit when they disagree. We report one such gate failing, in a cross-catalogue integrity study of two open registers of Earth-orbiting objects. A gate comparing a set-based Python path with SPARQL queries over the emitted RDF graph printed ALL CROSS-CHECKS AGREE on seven counts. Three were wrong, one overstated more than fourfold (932 against 220). Both paths imported the same constants, which encoded a misreading of the source's status vocabulary, so the error was common-mode and the gate could not see it. We give the mechanism, an object-level ledger reconciling every figure, and three checks that go back to the source's documentation, measured on the defective code and on its correction. We then checked that correction against each object's phase history, held in a source file the pipeline never read. The correction was also wrong: 42 of its 261 disagreements are artefacts, and none of our three checks flagged them. Finally, in a controlled replication with three pinned models and tools disabled, 72 of 75 paths generated on request as independent checks computed the defective count, 29 of 30 even when the prompt carried the source's own definitions of the codes. The evidence is one pipeline and one defect family. Within it, redundancy verified implementation, and the errors that reached publication were errors of meaning.
Abstract:A common safeguard for a data pipeline is redundant computation: derive each published number by two routes built on different technology and refuse to exit when they disagree. We report one such gate failing, in a cross-catalogue integrity study of two open registers of Earth-orbiting objects. A gate comparing a set-based Python path with SPARQL queries over the emitted RDF graph printed ALL CROSS-CHECKS AGREE on seven counts. Three were wrong, one overstated more than fourfold (932 against 220). Both paths imported the same constants, which encoded a misreading of the source's status vocabulary, so the error was common-mode and the gate could not see it. We give the mechanism, an object-level ledger reconciling every figure, and three checks that go back to the source's documentation, measured on the defective code and on its correction. We then checked that correction against each object's phase history, held in a source file the pipeline never read. The correction was also wrong: 42 of its 261 disagreements are artefacts, and none of our three checks flagged them. Finally, in a controlled replication with three pinned models and tools disabled, 72 of 75 paths generated on request as independent checks computed the defective count, 29 of 30 even when the prompt carried the source's own definitions of the codes. The evidence is one pipeline and one defect family. Within it, redundancy verified implementation, and the errors that reached publication were errors of meaning.
Accepted at the AI for Science workshop (NeurIPS 2026). Code, prompts, raw model outputs and per-trial records: this https URL (paper/gates/), archived at this https URL
Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Databases (cs.DB)
arXiv:2609.37603 [cs.SE]
(or arXiv:2609.37603v1 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2609.37603 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 288 words
Related coverage
Reviewing failure modes in science as we prepare for AI researchers
LessWrong · Oct 8, 2026
Why do models task game?
AI Alignment Forum · Aug 6, 2026
Our response to Séb Krier on Plan A
LessWrong · Jul 14, 2026
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026