analiza aiqbench
This research investigates a critical ambiguity in temporal memory systems where re-reading obsolete code can falsely trigger value restoration, drastically reducing model accuracy from 98.5% to 10.8%. The study demonstrates that distinguishing between an observation of a value and evidence of a genuine change requires specific guard mechanisms and provenance data beyond simple identity matching. These findings are vital for advancing reliable software verification and memory management in AI-driven development pipelines.
When Old Facts Return: Re-Reads, Reverts, and the Limits of Temporal Memory
arXiv:2610.07715v1 Announce Type: cross Abstract: A memory system can retire an obsolete value and later restore it merely because the same old statement appears again. A re-read of an old source and a genuine revert can produce the same observed sequence of values while requiring opposite current answers. We study this ambiguity on 130 extractor-selected atomic transitions derived from software fixes. In the ordinary transition condition, identity-based temporal memory reaches 98.5% model-judged accuracy with zero observed errors under a literal stale-value proxy. Appending a verbatim re-read of the old statement reduces accuracy to 10.8% and raises the stale-value rate to 88.5%. A guard that refuses to reactivate a previously retired value restores accuracy to 97.7% and reduces that rate to 0.8% in this constructed re-read condition. The guard cannot also recognize a legitimate revert without additional change provenance. Two supporting studies examine exposing retired history to the answer model and supplying current source for changed behavior. An exploratory extraction study over 707 software fixes provides scope context, not a universal coverage estimate. The design implication is to distinguish an observation of a value from evidence that the value changed. Selected inputs, aggregate-only answer records, related-family judges and a post-failure guard evaluation limit the conclusions to the retained experiments.
Abstract:A memory system can retire an obsolete value and later restore it merely because the same old statement appears again. A re-read of an old source and a genuine revert can produce the same observed sequence of values while requiring opposite current answers. We study this ambiguity on 130 extractor-selected atomic transitions derived from software fixes. In the ordinary transition condition, identity-based temporal memory reaches 98.5% model-judged accuracy with zero observed errors under a literal stale-value proxy. Appending a verbatim re-read of the old statement reduces accuracy to 10.8% and raises the stale-value rate to 88.5%. A guard that refuses to reactivate a previously retired value restores accuracy to 97.7% and reduces that rate to 0.8% in this constructed re-read condition. The guard cannot also recognize a legitimate revert without additional change provenance. Two supporting studies examine exposing retired history to the answer model and supplying current source for changed behavior. An exploratory extraction study over 707 software fixes provides scope context, not a universal coverage estimate. The design implication is to distinguish an observation of a value from evidence that the value changed. Selected inputs, aggregate-only answer records, related-family judges and a post-failure guard evaluation limit the conclusions to the retained experiments.
12 pages, 1 figure. Ancillary files contain retained aggregate evidence, derived scenario and annotation exports, reference code, and an offline verifier
Software Engineering (cs.SE); Computation and Language (cs.CL)
arXiv:2610.07715 [cs.SE]
(or arXiv:2610.07715v1 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2610.07715 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 246 cuvinte
Subiecte conexe

The payload: the semantic layer, memory, and the loop that fills it
Neo4j · 10 aug. 2026

Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · 3 aug. 2026
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · 12 iun. 2026

PPA 11: The Basic Model
LessWrong · 14 oct. 2026