aiqbench analysis
Composing insight from the full article — this may take a moment.
Literal Lock-In: How Test Language Shifts Literal Attraction in Typed Decision Models on Bengali Idioms
arXiv:2610.22526v1 Announce Type: new Abstract: Decision models, non-generative scorers that report a probability over caller-supplied options and emit no text, are increasingly deployed as routers, classifiers and guardrails. Because they generate nothing, rationales and chain-of-thought do not apply. We audit three: an encoder scorer, a decoder scorer and a closed API, entirely through interventions on their input, using expert-annotated Bengali idioms carrying a gold figurative and a gold literal gloss. Dropping an idiom's own literal gloss into the option list draws every arm toward it, taking 65% to 96% of each arm's remaining Bengali errors. A model-by-language matrix over one shared item set shows how far this depends on the test language: every arm is 1.9 to 6.4 times more captured in Bengali than in English on the same paired entries, the commercial model's apparent resistance is a property of its test language, and capability does not order the failure. Calibration degrades in all three arms, and on the open arms confidence inverts, so gating on it is worse than gating at random. On a semantics-free option list, where accuracy is at chance, the best-calibrated arm still reports mean confidence 0.82. Code and data: https://github.com/sakhadib/literal-lock-in.
Read the full article at arXivRelated coverage
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026

[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · Sep 17, 2026

I Did This Without Your Knowledge
LessWrong · Oct 17, 2026

Why the current tech backlash feels different
The Verge · Sep 10, 2026