analiza aiqbench
This benchmark analysis reveals a nuanced performance landscape where Opus 5.5 leads in overall trajectory completion but exhibits specific edge-case failures, while Opus 5 demonstrates superior reasoning capabilities that Fable 5.1 lacks. The data highlights distinct failure modes across model generations, with Opus 5's primary weaknesses rooted in faulty inference rather than operational execution, offering critical insights for developers optimizing coding agents.
Opus 5.5 vs Opus 5 vs Fable 5.1: Coding Benchmark Results
We’ve now run three generations of frontier models through the same expert-created terminal-bench style task set: Fable 5.1 in September, Opus 5 in July, and, now Opus 5.5. For this analysis, the task set contained 200 trajectories, of 24 tasks, with every failure traced to a judge-confirmed root cause. A generational climb On the task set, pass@1 is 61.5% for... The post Opus 5.5 vs Opus 5 vs Fable 5.1: Coding Benchmark Results appeared first on Snorkel AI.
We’ve now run three generations of frontier models through the same expert-created terminal-bench style task set: Fable 5.1 in September, Opus 5 in July, and, now Opus 5.5. For this analysis, the task set contained 200 trajectories, of 24 tasks, with every failure traced to a judge-confirmed root cause. On the task set, pass@1 is 61.5% for Fable 5.1, 60.7% for Opus 5, and 60.7% for Opus 5.5. This is a flattening out of this score across the last two generations, with Fable 5.1 nominally higher than…
Extras afișat conform termenilor RSS ai publicației · 87 cuvinte · textul integral și figurile sunt pe site-ul publicației
Subiecte conexe
[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model
Latent Space · 13 oct. 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026
[AINews] Claude Fable/Mythos 5.1: new SOTA model, 75% cache price cut but 70% more output tokens
Latent Space · 2 sept. 2026
[AINews] Claude Opus 5.5, the new default model for AINews — and everybody cuts prices 40-50%
Latent Space · 23 sept. 2026