Is Jev as Accurate as Frontier Models at Classification?

Jev is TypeSafe’s System One decision model. Give it an app state object and a typed question, and it gives you back a typed answer, a confidence score, and the probability for each possible option (the Decisions API response type, not the chat endpoints). According to the TypeSafe pitch, you can replace a frontier chat model with a small judgment model like Jev and get significant cuts in both cost and latency, on classification tasks such as this one. How much accuracy are you sacrificing? We ran 3,080 Banking77 customer support utterances through both Jev 1.13 and Claude Opus 5, the model OpenRouter users spent the most on for classification in the Task spend section of its rankings page as of 22 September 2026. Each utterance gets sorted into one of 77 banking intents, and both models got the same one-line description of each intent. The chart below puts Jev and Opus side by side, showing how they compare on accuracy, median latency,…