GPT-6 Astra: the harness now matters more than the model

GPT-6 Astra is out, and the number worth your attention is not the one OpenAI put on the slide. Simon Willison posted the rollout details on 3 September 2026, and buried in the ARC-AGI results is the same model scoring 99.9% and 62.7% on the same benchmark.

The difference was not the model. It was the harness wrapped around it. That gap is the most useful thing any of us learned this week.

---

🔍 One model, two scores, and the cheap run won

Here are the two ARC-AGI 3 runs as reported:

Run	Harness	Score	Cost
OpenAI's submission	Custom "Provider Adapter" harness	99.9%	$19,000
ARC-AGI's own run	Default harness	62.7%	$26,000

Read the cost column twice. The higher-scoring run was also the cheaper one. A 37-point swing in capability, at 27% less spend, from changing nothing but the code that feeds the model its problems and reads back its answers.

Key takeaway: On this benchmark, the scaffolding was worth more than any model upgrade OpenAI shipped this year. You cannot buy a better model than your competitor — you both pay $10 per million input tokens. You can absolutely build a better harness, and that part is free.

This is not an argument that OpenAI cheated. Custom harnesses are disclosed and allowed. It is an argument that published benchmark numbers now describe a system, not a model, and you cannot read one off a chart and expect to reproduce it from a naive API call.

---

💰 Frontier pricing has converged, and that changes the decision

Astra is priced at $10 per million input tokens and $50 per million output tokens — identical to Claude Fable 5 and 5.1.

Model	Input / 1M	Output / 1M
GPT-6 Astra	$10	$50
Claude Fable 5	$10	$50
Claude Fable 5.1	$10	$50

Two labs, same sticker price, to the dollar. That tells you something structural:

• Price is no longer a differentiator at the top tier. Nobody is undercutting anybody.
• Model choice is now a fit decision, not a budget decision. Pick on eval results for your task, not on a per-token line item.
• Your bill is decided by your token count, which is decided by your prompt design, your caching, and your retry policy. All of which you control.

If you are quoting client work from Sri Lanka, price that in rupees before you commit rather than after the invoice arrives — the USD figure flatters itself. Our LKR exchange rate page has today's rate if the one in your head is six months old.

---

📊 Self-reported wins, independent draw

OpenAI's own numbers show Astra comfortably ahead of its predecessor, GPT-5.6 Sol:

Benchmark	GPT-6 Astra	GPT-5.6 Sol
ExploitBench	100%	78.5%
ExploitGym	42.4%	30.3%
SRE-Bench (4 attempts)	99.2%	68.7%
Eight-needle recall, 256K–512K ctx	100%	—
Eight-needle recall, 512K–1M ctx	96.3%	—

Then there is the independent scoreboard. On the Artificial Analysis Intelligence Index:

Model	Intelligence Index
Claude Fable 5.1	66
GPT-6 Astra	61
GPT-5.6 Sol	61

Astra ties its own predecessor on the independent aggregate while beating it decisively on the vendor's own suite.

Both things can be true. Vendor benchmarks are chosen because the model does well on them; that is not fraud, it is marketing, and every lab does it. But when a self-reported suite shows a large jump and a third-party aggregate shows a flat line, the honest read is: the gains are real but narrow. Security tasks and long-context recall look genuinely better. General reasoning, on the outside measure, did not move.

Worth noting on the cost side: Willison reports Astra sitting on the cost-efficiency frontier of the Artificial Analysis Coding Agent Index, scoring roughly two points above Sol there. Cheaper-per-unit-of-coding-work is a less exciting headline than 99.9%, and a lot more relevant to anyone actually paying the bill.

---

🛠️ What to do with this if you have no budget

The harness result is the gift here, because harness work costs engineering time rather than API credits. If you are a student, a freelancer, or a two-person team, these are the moves that pay:

1. Build an eval set before you pick a model. Twenty to fifty real examples from your actual use case, with known-good answers. This is a spreadsheet, not a platform. It will tell you more than any leaderboard.
2. Test your harness, not just your prompt. How you chunk input, how many attempts you allow, how you parse output, what you do on a malformed response — the ARC-AGI gap says that layer carries real points.
3. Measure retries honestly. A "4 attempts" score is a 4× bill. Decide deliberately whether your product can afford that, per request.
4. Cache aggressively. At $10 per million input tokens, repeated context is the single easiest line item to cut.
5. Do not migrate on announcement day. Astra is still rolling out — OpenAI's own wording is that access expands "over the coming days." Let others find the rough edges.

I have not used Astra yet, and neither had Willison when he wrote that post. Nobody publishing a verdict today has run it against a real workload. Treat every confident take you read this week, including the enthusiastic ones, as speculation with good formatting.

---

💡 What this means for you

Frontier models are converging into a commodity tier: same price, similar aggregate scores, different shapes of strength. That is good news if you are building from Colombo or Kandy on a personal card, because it means the expensive part of the advantage is no longer purchasable.

Bottom line: Everyone with $10 buys the same million input tokens. What separates a working product from a demo is the eval set, the harness, and the caching — and none of those have a price list.

Spend this week writing twenty test cases for whatever you are building. That will age better than switching model strings.