Jev: Decision Models on Trial

How does a question-answering architecture stack up against traditional classification on content safety tasks? Jev’s been making the rounds lately, and for good reason — instead of generating prose, it answers named questions directly with a probability or score, which is a neat trick if you’ve spent any time wrangling classifier outputs. Naturally, that got us curious: how does this new breed of decision model actually hold up against a traditional, purpose-built classifier? So we ran a quick comparison — new decision models versus a custom-trained classifier on content safety tasks. Here’s what we found. Every AI safety classifier runs on policy labels, and those labels have to come from somewhere — people, models, or both. The common shortcut is an LLM-as-judge: hand it a safety policy and a…