analiza aiqbench
Mercury Decide matches Jev on claim verification and beats it on LegalBench, at a quarter of Jev's price and with lower error at the 1% budget. Jev is still faster, and Mercury Decide trades some accuracy on direct-support claims for gains on subtle contradictions. Readers choosing a verification model should weigh cost and the category mix against their own workload, and the gaps on LegalBench deserve testing on their own data.
An Evaluation of Inception's Mercury Decide
Compare Mercury Decide, Jev and frontier LLMs on claim verification and a LegalBench subset, with accuracy, cost, latency and calibration results.
Earlier this week, we published an independent evaluation of TypeSafe’s Jev. We ran Inception’s new model, Mercury Decide, through the same tests. Mercury Decide works like Jev: you input a state and a list of allowed answers, and it returns one answer with a probability for every option, in a single call. Inception prices it at half of Jev’s rate, $0.021 per million input tokens, and output is free. The datasets, prompts, scoring, and statistics for this…
Extras afișat conform termenilor RSS ai publicației · 77 cuvinte · textul integral și figurile sunt pe site-ul publicației
Subiecte conexe
Adaptive Model Inversion Attacks Generalize a Privacy-Robustness Tradeoff
arXiv · 7 oct. 2026
Taming foundation model with invariance-oriented pre-training for broad-spectrum EEG analysis across signal-level, brain-state, and brain-health tasks
arXiv · 27 aug. 2026
Assessment of Conditional Diffusion Model for Synthetic Histopathology Image Generation
arXiv · 5 aug. 2026
SFA: A Statistical Physics-Inspired Diffusion Transformer for Efficient and High-Quality Image Generation
arXiv · 4 nov. 2026