Blog · Analysis
From 30.2% to 96.2% — the gap is the harness
August 20, 2026 · dshbase · rewritten analysis
August’s DeepSeek news cycle wanted you to memorize price and parameter counts. The more durable signal is sitting next to the model: DeepSeek Harness — the open runtime that decides whether those tokens become a daily workflow or a demo.
Rewritten from WeChat analysis that framed V4 Pro and Harness on the same table (e.g. the “30.2到96.2” thread). Numbers below are cited as reported in that coverage and public write-ups; treat vendor-only benches with the usual skepticism.
The model story is real — and incomplete
V4 Pro shipped with long context, dual API shapes (OpenAI / Anthropic-style), tool calling, and aggressive list prices. Cache hits matter even more for agent workloads: you re-read history constantly. That pricing story moves bills. It does not, by itself, move how an agent recovers from a failed tool call or how your team replays a bad session.
30.2 → 96.2: weights frozen, environment changed
The ARC-AGI-3 anecdote that keeps circulating: Opus-class baseline around 30.2% on ARC Prize’s framing; developer Jeremy Berman’s setup — a Claude Code-like environment, an action channel, a filesystem log — let the model write parsers, search, and simulators across the public tasks and land near 96.2% (24/25 in one reported pass). Hundreds of programs, tens of thousands of lines — still the same weights.
What jumped was not “IQ on a card.” What jumped was what the model was allowed to do between guesses: persist state, iterate, instrument the problem. Humans hear “the model got smarter.” Operators should hear “the runtime opened the action space.”
Why Harness is the sticky layer
Frontier scores will keep chewing each other (CyberGym, Terminal-Bench, CursorBench, office suites — pick your favorite weekly leaderboard). Prices will compress. A runtime that absorbs your plugins, permissions, project layout, cache policy, and failure replay grows switching cost. You can swap the upstream model; you rarely throw away the way your team already works.
DeepSeek Harness leans into that: Cordis seams for tools/policy/storage/context, Trajectory for event-level replay, visible token/cache meters, community plugins that can go deeper than “add an MCP.” UI polish still trails Codex-class shells — early ecosystems often look ugly on purpose.
How we use that insight at dshbase
- If you are tasting V4 Pro: check price, API shape, tool calling, and whether it slots into tools you already pay for.
- If you are building a lasting agent workflow: demand session replay, permission boundaries, cache visibility, and a plugin story you can audit — start with trajectory tracing and cost & caching.
- If you install community plugins: treat “works on my machine” as the product. Our directory only marks verified after a real install path, because a pluggable runtime without install truth is just a longer README.
Takeaway
Model launches publish a ceiling. The harness decides whether you can push that ceiling into Tuesday afternoon work. Cheap models are plentiful; runtimes that turn models into systems are not. The next round to watch is not only the next third-party bench — it is whether a handful of DSH plugins become load-bearing for ordinary developers.