The ghost model that "crushed" GPT-5.6 — and the statistics lesson that came with it

What showed up

On August 20, 2026, a model with no company name attached appeared on OpenRouter under the identifier stealth/ox-alpha. At the same time, it showed up in OpenCode, a terminal-based coding agent.

The specs drew attention on their own: 1,048,576 tokens of context, up to 131,000 output tokens per call, text, image, and video input, native support for tool calling and structured JSON output. And free for a week.

Models like this have a name: stealth models. They are put into public testing under a cover name, with the maker never stepping forward. The practice is legitimate and has logic to it — you can measure real preference without the expectation bias a known brand carries, and you can harvest real usage before an official launch. It is also, conveniently, free marketing.

The number that went viral

In early independent tests, Ox Alpha closed out the DeepSWE coding benchmark at 80%, against 65% for Claude Fable 5 and 52% for GPT-5.6-sol.

Within 24 hours the internet had decided: mystery model crushes the leaders.

The detail almost nobody read

That 80% came from an initial sample of ten tasks.

A later evaluation, on a larger subset, produced roughly 63%.

Ten tasks. Let us be concrete about what that means: at n=10, each individual pass or fail moves the result by a full ten percentage points. Eight out of ten is statistically compatible with a true success rate across an enormous range — something on the order of 44% to 97%, depending on which interval you use.

Translated: that 80% was not measuring the model. It was measuring the coin flips.

The model may well be excellent. 63% on a larger sample is still strong. But the headline that circulated was built on a measurement that supported none of the comparisons it made.

The checklist for not falling for the next one

Ox Alpha will get a name, and two weeks from now another ghost appears. The five questions that save your week:

1. What is the n? If the source does not say how many tasks were evaluated, the number is not a result — it is an anecdote with decimal places.

2. Who ran it? A vendor's own evaluation, an independent evaluation, and a screenshot on social media are three different things with three different confidence levels.

3. Is the benchmark in the training data? Data contamination is the field's quiet problem. A public, well-known benchmark tends over time to become training data — and then the score measures memory, not capability.

4. What is the cost per solved task? A raw percentage hides economics. A model solving 63% at a tenth of the price beats, in production, one solving 70% expensively. The business metric was never the percentage.

5. Did it reproduce on your code? Benchmarks run on public, well-documented repositories with tests. Your legacy code is not like that. The distance between those two worlds is where promises die.

The test worth more than any leaderboard

Thirty minutes, once, and you stop depending on headlines forever:

1. Pull twenty real, already-closed tasks from your backlog of the last three months — issues that became actual PRs.
2. Keep the original PR as the answer key.
3. Run the candidate model on the same tasks, with the same context a developer would have had.
4. Measure four things: did it actually solve it, did it pass CI, what did it cost, and how much human review time did it consume.

That set ages well, works for the next model and the one after, and answers the only question that matters: not "which model is best in the world?" but "which model is best for my code, on my budget?"

The last line

Public benchmarks tell you who is in the race. That has value — it is the filter deciding who is worth testing.

But no leaderboard knows your legacy code, your naming conventions, your strange tax rule, or that module nobody has understood since 2019. Only your repository knows who actually works for you. Twenty real tasks are worth more than ten thousand posts about the model of the week.