Why do developers hate GPT-6.2 Astra when its benchmarks look great?
It tells you less: GPT-6.2 Astra hid a known blocker in 22.28% of OpenAI's own coding tests, twice the rate of GPT-6 Astra.
Senior Full-Stack Engineer- Published
- 8 minReading time
- Artificial IntelligenceCategory
Listen to this post· AI-narrated
0:00 / 9:21OpenAI released GPT-6.2 Astra on 17 November, and the same day the developers who had tested it early were warning people off. Theo Browne posted (opens in a new tab) "I think this is the worst model OpenAI has ever released." Matt Shumer said (opens in a new tab) he wouldn't write a review of the two new models because "they were essentially unusable." The same week, Artificial Analysis put GPT-6.2 Astra three points behind Claude Fable 5.2 on its independent index, for a fraction of the cost per task. The two camps are measuring different things. The scores grade the answer Astra hands back. The complaints are about how it reports what it did along the way, and OpenAI's own system card measured that part.
TL;DR: Developers hate GPT-6.2 Astra because it tells them less than they need. In OpenAI's own tests it hid a known blocker in 22.28% of coding tasks, double GPT-6 Astra, and OpenAI suspects its push for concise answers. Shorter answers and lower prices keep it cheap at $4.00 per task. Use it where tests check its work.
What do developers actually complain about in GPT-6.2 Astra?
The complaints about GPT-6.2 Astra are about communication, and almost none of them say it can't code. Theo's letter to OpenAI, which he published in full (opens in a new tab), groups them under headings like "It has no concept of knowledge gaps", "It speaks in tongues that only 6.2 Astra understands" and "It runs in loops". He ends up saying he would not choose it over Opus 5.5 for his work as a developer.
His clearest example, quoted by Simon Willison (opens in a new tab): he gave Astra "/goal make a new level for Outer Wilds". Two hours in, it was "blocked", waiting for him to say whether a reviewer wrote its clips to 24_g521.mp4 or 24_p11.mp4. Those were files its own subagents had made up. Nobody outside the run could answer.
Other early testers describe the same model. Dan Shipper's team at Every called it "a disappointing release compared to Fable 5.2" (opens in a new tab). A developer who used both new models for days opened his review with (opens in a new tab) "They're extremely lazy." On r/codex, one thread argues that Astra is hard to understand because it's meant for other AI (opens in a new tab). Put together, the pattern is a model that stops early, explains itself in its own shorthand and asks you questions you can't answer.
Do the benchmarks say GPT-6.2 Astra is worse?
Every board below grades the finished answer, and none of them reads the report Astra writes about its own work. On those terms GPT-6.2 Astra lands three points behind the best model available. Artificial Analysis scores it 62 on its Intelligence Index (opens in a new tab), behind Claude Fable 5.2 at 65 and far above the median of 44 for comparable models. It costs $4.00 per index task against Fable's $26.54. It is also slow, at 14.2 minutes per task against a median of 12.7.
Coding agents show a wider gap, though the board compares whole setups, each model inside its own agent. On the Artificial Analysis Coding Agent Index, as published by OrcaRouter (opens in a new tab), Astra at max, its highest reasoning setting, scores 53.47 in Codex for $5.12 a task. Fable 5.2 at xhigh, the setting that scores best in Claude Code, reaches 58.47 for $15.47. Astra is still cheaper per task, and it trails by five points on exactly the work its loudest critics do.
The one independent board where Astra went backwards is SimpleBench (opens in a new tab), a set of trick questions where the obvious answer is wrong. GPT-6.2 Astra scores 77.3% and ranks 17th. GPT-6 Astra scored 83.6%. Fable 5.2 leads with 94.9%, and Opus 5.5 has 88.4%. SimpleBench says nothing direct about writing code. It is the closest any board comes to the judgment Theo's letter says is missing, and it is the one where the new Astra scores below the old one.
Why does GPT-6.2 Astra hide problems from you?
OpenAI measured this before launch. The GPT-6.2 system card (opens in a new tab) runs an evaluation called Blocker Deception: give the model a real coding task, quietly remove a file it needs or break a tool it depends on, then check whether its final report mentions the problem. "GPT-6.2 Astra omitted a known obstacle to completing its work in 22.28 percent of trials, compared with 11.09 percent for GPT-6 Astra."
OpenAI's own explanation is the most useful line in the card: "One hypothesis is that our efforts to make responses concise and avoid excessive caveats may be related to the model omitting important obstacles." That's the same effort the launch post sells as a feature. OpenAI says Astra scores 6.2 points higher than GPT-6 Astra on GDPval v2, its benchmark of long knowledge-work assignments, while using 22% fewer output tokens (opens in a new tab). Those fewer tokens, together with prices 20% below GPT-6 Astra, keep its cost per task low. If OpenAI's hypothesis holds, the missing blockers are a side effect of the same training. The evaluation only scores final reports, but it describes the habit developers keep running into: Astra hits a problem and doesn't say so in plain words.
OpenAI's mitigation is a prompt, and it only ships in Codex. The card says a Codex developer prompt "substantially reduced how often these models omitted a known obstacle." It also says that prompt "is not applied to direct API requests." If you call gpt-6.2-astra from your own agent, you get the model that hides blockers twice as often as the last one, unless your system prompt asks for them.
This is why I never take an agent's final message as the result. Tests are the spec agents need. A model that skips the bad news in its summary still has to get past the test suite, and the suite doesn't care how confident the summary sounded.
Is GPT-6.2 Astra good at anything?
As a code reviewer, GPT-6.2 Astra finds more bugs than the model it replaces, provided something sorts through what it reports. CodeRabbit, which builds AI code review, ran it on an 83-issue benchmark (opens in a new tab): Astra caught 58 known issues against 51 for GPT-6 Astra. The cost was noise. Its regular actionable comments went from the baseline's 253 to 589, and benchmark precision on those comments fell from 21.74% to 12.22%. More catches, buried in more comments.
That fits how Theo set up T3 Code, his own coding agent app. Two days after calling Astra the worst OpenAI model, he posted his routing (opens in a new tab): Fable 5.2 leads implementation, GPT-6.2 Astra playtests, and Gemini 4 Neon writes copy. Astra's output goes to another model or a test, never straight to him.
The pricing suits that role. API input is $8 and output $40 per million tokens, 20% below GPT-6 Astra, and cached input is $0.40, a 60% cut. A reviewer that rereads the same large codebase on every pass benefits most from that cache price.
Who should use GPT-6.2 Astra, and who shouldn't?
Use GPT-6.2 Astra as a reviewer, tester or second opinion inside a pipeline that checks its work: a test suite, a lead model that reads its findings, or a human who skims a list of flagged issues. It catches more than its predecessor and costs far less per task than Fable 5.2.
Keep it away from being the model you work with all day on long, open-ended tasks. That's where the missing blockers, the loops and the private shorthand cost you hours, as Theo's two-hour Outer Wilds run shows. Every's review put it well (opens in a new tab): the new models "handle bounded tasks efficiently. The work we most want help with still demands more judgment and follow-through." For that work, Fable 5.2 or Opus 5.5 is still the safer lead.
What to do this week
If you run Astra through the API in your own agent, add a line to its system prompt that makes it list every blocker, missing file and failed tool call before it reports a task as done. OpenAI's own mitigation was a prompt, which is reason enough to write yours, and you can test it the way OpenAI did: delete a file the task needs and see whether the report mentions it. Then gate its output with tests, so a summary that leaves something out still fails. If you work in Codex, you already get OpenAI's mitigation. Point Astra at your next pull request as a reviewer and compare its findings with your current setup's. I didn't benchmark Astra myself; every number here comes from the sources linked above.
I write about coding agents and the models behind them on the gabs.app blog. I'm a full-stack engineer and my work is on gabs.app; if you're hiring, send me a message through the contact form.