# A cheap 'decision model' that asks Opus only when unsure matched Opus at 4.4x lower cost. What Hightouch's test shows

> Hightouch graded record-matching with TypeSafe's Jev and escalated only the uncertain middle band to Claude Opus 5.5, getting within 0.2 points of Opus on easy cases at under a quarter of the cost. On hard cases the saving fell to about 1.8x, and the whole test ran on synthetic data, but the reason it worked is useful for anyone building model cascades.

Published Nov 7, 2026 by TechMeasured Staff · AI · Canonical: https://techmeasured.com/ai/hightouch-jev-escalate-to-opus-hybrid-judge-4x-cheaper-what-the-numbers-show

Hightouch says a small "decision model" that hands its uncertain cases to Claude Opus 5.5 can match Opus's accuracy at grading identity-resolution results while costing about a quarter as much. The November 4 post, by Albert Jin, Neil Desai and Delaney Mackenzie, drew more than 560 points on Hacker News this week.

Why it matters: lots of teams now run a cheap model first and escalate to a frontier model. Hightouch's numbers show that what makes this pay off isn't the cheap model's raw accuracy. It's whether the cheap model knows when it's likely to be wrong.

## What was tested

Identity resolution merges records that seem to describe the same person. The common failure is an over-merge: two real people collapsed into one. Hightouch asked each model to judge whether an identity was over-merged, a job it says humans do at about 50 identities an hour.

The data was synthetic, generated with pseudopeople, a University of Washington IHME tool that simulates a fictional US town, with realistic errors added. There were two 1,000-identity tests: an easier mix (half clean single people with noisy records, half over-merges) and a harder set of tricky cases such as twins, juniors and seniors, and people with the same name.

Jev, from TypeSafe, and OpenAI's GPT-6 Luna (via its Decisions API) are "decision models": instead of writing text, they return a 0–1 confidence that a condition is true. In the hybrid setups, the decision model graded every identity and only those in a middle confidence band went to Opus.

## The numbers

Easier test, per Hightouch:

| Setup | Accuracy | Cost per 1,000 identities |
|---|---|---|
| Claude Opus 5.5 | 99.0% | $3.86 |
| Gemini 3.1 Pro | 98.3% | $4.48 |
| GPT-6 Sol | 98.3% | $2.50 |
| GPT-6 Luna alone | 97.3% | $0.10 |
| Jev alone | 95.7% | $0.05 |
| Hybrid Luna (6% sent to Opus) | 98.5% | $0.33 |
| Hybrid Jev (22% sent to Opus) | 98.8% | $0.88 |

Harder test:

| Setup | Accuracy | Cost per 1,000 identities |
|---|---|---|
| Claude Opus 5.5 | 95.1% | $5.39 |
| GPT-6 Luna alone | 88.8% | $0.12 |
| Jev alone | 89.1% | $0.06 |
| Hybrid Luna | 91.6% | $1.15 |
| Hybrid Jev | 95.3% | $2.94 |

On the easier test hybrid Jev cost about 4.4x less than Opus alone; on the harder one, about 1.8x less, as FourWeekMBA calculates. The saving shrinks because more cases land in the uncertain band, so the bill scales with difficulty. Speed follows the same pattern: Hightouch reports roughly 0.15 seconds per identity for Jev alone, 2.2–2.6 seconds for Opus, and 1.56 seconds for hybrid Jev on the harder test.

## Why the less accurate model won

On the easier test, Luna beat Jev on its own (97.3% vs 95.7%). Yet Jev made the better hybrid, and on the harder test it was the only setup to match Opus.

Hightouch's explanation is calibration. Jev's mistakes mostly sat near 50% confidence, exactly where the hybrid sends cases to Opus. Luna's mistakes often came with 90–100% confidence, so they skipped escalation and stayed wrong. On the harder set, the authors say, Luna was badly overconfident that over-merges existed.

Not everyone agrees on how clean that result is. On Hacker News, one commenter estimated Luna's calibration error was actually slightly lower and asked Hightouch to publish expected calibration error figures rather than bar charts.

This fits an earlier, separate benchmark. In September, Arize reported that Jev matched Claude Opus 5's 87% accuracy on a hallucination-detection test once thresholds were tuned, at a fraction of the cost and latency. Arize sells observability tools, and it ran that test itself.

## The caveats

- **Synthetic data, one task.** Hightouch hasn't validated this on real customer records, and FourWeekMBA notes there are no confidence intervals.
- **Tuned on the test.** The cutoffs and escalation bands were picked on these same test sets, which can flatter results on new data.
- **Vendor setup.** Costs and latencies come from Hightouch's own pipeline. No code was released, though the post includes its full prompt.

## What to take from it

If you're building a cascade, test the cheap model's calibration before its accuracy: plot confidence on the cases it gets wrong. A model that's unsure when it's wrong makes a good gatekeeper; one that's confidently wrong will quietly pass errors through no matter how good its headline score is. Hightouch says follow-up posts will cover under-merging and open-weight decision models.


## Sources

1. [Hightouch — LLM Judges for Identity Resolution at 4x Lower Cost](https://hightouch.com/blog/llm-judges-identity-resolution)
2. [FourWeekMBA — Cheap Model Plus Opus Nears Opus Accuracy at 4.4x Less Cost](https://fourweekmba.com/ai-cheap-model-plus-opus-nears-opus-accuracy-at-4-4x-less-cost/)
3. [Arize — Jev vs. LLM-as-a-Judge: Accuracy and cost benchmarks](https://arize.com/blog/jev-as-a-judge/)
