Short answer: for one million classifications of about 500 tokens each, Jev costs roughly $21, Claude Haiku 4.5 roughly $550, and a rule you write yourself costs nothing. Jev is the fastest managed option and cannot return a malformed answer; Haiku is more accurate on a single broad question and tells you why; a fine-tuned classifier beats both on a stable task if you have labelled data; and a regex still beats everything on signals that are genuinely simple. Choose on explainability and accuracy on your data, not on the price list.
The comparison at a glance
| Option | Cost per 1M decisions* | Latency | Explanation | Customisation | Time to first result | Best at |
|---|---|---|---|---|---|---|
| Jev (decision model) | $21 | 70–500ms | None | None: same weights for everyone | Minutes: an API call | High-volume routing, triage and guardrails |
| Claude Haiku 4.5 | $550 | Typically ~1–3s | Yes, in words | Prompt and examples | Minutes | Mixed work where you also need text or a reason |
| Claude Sonnet 5 | $1,100 | Typically a few seconds | Yes, in words | Prompt and examples | Minutes | Harder judgements, still needing an explanation |
| Fine-tuned BERT-style classifier | Your own GPU or CPU time | Milliseconds, self-hosted | None (scores only) | Full: trained on your labels | Days to weeks, plus labelled data | Stable, high-volume tasks you own end to end |
| Rules or regex | Effectively zero | Microseconds | Total: the rule is the reason | Total, by hand | Hours | Clear signals, and as a baseline you must beat |
*Assumes ~500 input tokens per decision (500M tokens) and a short output where one applies. Jev at $0.042/MTok input, output free. Claude at list prices: Haiku 4.5 $1/$5 and Sonnet 5 $2/$10 per million input/output tokens, September 2026.
What published tests actually show
There is no large independent benchmark suite for this model class yet, so treat everything below as directional. These are third-party results reported since launch, not our own measurements.
| Test | Jev | Comparison | Worth noting |
|---|---|---|---|
| Phishing, single question, 2,000 synthetic emails | Jev 62.6% | Haiku 4.5 81.3% | A two-line regex scored 91.8% |
| Phishing, five questions combined, 1,000 held-out emails | Jev 95.0% | Haiku 4.5 93.2% | Haiku's best single signal, 94.2%, beat its own multi-question version |
| Spam, 18,514 emails | Jev 98.33% | TF-IDF baseline 98.39% | The cheap classical baseline matched it |
| Event screening, 50 cases | Jev 96% | Gemini 3.5 Flash-Lite 86% | Small sample; treat as directional |
| Tool-call risk, 60 cases | Jev 91.7% | — | Repeated runs gave 91.7% and 93.3%, so expect some variance |
Source: third-party evaluations reported in September 2026. Sample sizes are small and several datasets were synthetic.
Two patterns survive the caveats. First, question design decides the outcome: the same model went from 62.6% to 95.0% on the same task purely by splitting one question into five. Second, the boring baseline is dangerous: a regex and a TF-IDF model each matched or beat a frontier-priced setup on their home ground. Any comparison that skips the baseline is marketing.
How to choose, in four questions
Picking the right tool for a repeated decision
- 1Will you ever have to justify the decision?If yes, you need words. That rules out Jev and a BERT classifier for the final call, though either can still triage what a human reviews.
- 2Is a simple rule genuinely enough?Try it first, and measure. If a regex gets you 92% on a task where mistakes are cheap, a model has to beat that before it earns its keep.
- 3How many decisions per month, and how fast?Below roughly 100,000, cost barely matters, so pick on accuracy and explanation. Above that, or below a second of latency, a decision model starts to pay.
- 4Do you have labelled data and someone to own a model?If yes, a fine-tuned classifier is cheaper to run than anything hosted and you control it. If no, the hosted decision model is the shortcut.
Where each one genuinely wins
Jev wins when the decision is high-volume, latency matters, the output must be one of a fixed set, and nobody needs a reason. Gating tool calls inside an agent is the clearest case: it has to happen in milliseconds, thousands of times, and the answer is a yes or a no.
Claude Haiku or Sonnet wins when the same call must also produce text, when you need the reasoning for a human or an audit file, when the task shifts often enough that rewriting a prompt beats redesigning a question set, or simply when your volume is too low for cost to matter. Our frontier model comparison covers picking between those.
A fine-tuned classifier wins on a task that will not change for a year, where you already have thousands of labels, and where per-call cost at scale matters more than setup time. Old encoder models are still downloaded tens of millions of times a month for exactly this reason.
Rules win more often than anyone admits. They are free, instant, perfectly auditable, and easy to fix. Use them as the baseline every model has to beat.
The combination most teams will land on
Not one of these, but a chain: a rule filters the obvious cases, a decision model handles the rest in milliseconds and says how confident it is, anything below the confidence threshold goes to a frontier model or a person, and every outcome gets logged. That structure is cheap, fast, and still explainable where it counts, because the expensive model only sees the cases that are genuinely hard.
Building that chain is ordinary engineering: routing, thresholds, evaluation and logging. It is what we teach on our AI Engineering programme, and the reason we tell teams to write the evaluation set before choosing the model.
Before you switch anything: label 300 to 500 of your own real cases and run every candidate against them, including a plain rule. Every number in this article, ours and TypeSafe's, came from someone else's data. Yours is the only benchmark that decides the question.
Three days building production AI systems: routing, evaluation, guardrails and cost control across models, not just prompts.
Next public class: 11–13 November 2026 at Worq KL Eco City · seats available
HRD Corp claimable. AITraining2U is an HRD Corp registered training provider. Our public and in-house courses are claimable under the HRD Corp Claimable Courses (SBL-KHAS) scheme, and we prepare the grant paperwork with you. See HRDC-claimable AI training: courses, costs and how to claim.