GPT-6 Astra Benchmarks Explained
OpenAI released GPT-6 Astra on September 3, 2026, and it did not hedge. The announcement calls Astra "the world's most intelligent and aligned model," and Greg Brockman told reporters it is "not unreasonable to feel that we are now in the AGI era." Under the hood this is OpenAI's largest training run ever, over 100,000 GPUs at the Stargate site in Texas, and the first release where earlier OpenAI models supervised the training of the new one.
The benchmark tables back most of the swagger. Astra posts the highest scores OpenAI has ever published on abstract reasoning, math, and cybersecurity. It is also the first model to hit the Critical cybersecurity threshold under OpenAI's Preparedness Framework, which is why it is rolling out slowly, with advanced cyber capabilities gated behind the Daybreak program. And not every table goes OpenAI's way: Meta's Muse Spark 1.3 edges Astra on agentic coding, and the Artificial Analysis index has Claude ahead. Here is every category OpenAI published, in the order OpenAI published it.
1. Computer use: Agents' Last Exam, OSWorld 2.0, ScreenSpot-Pro
Computer use is the headliner. Astra can drive a desktop, fill out forms, update CRMs, run frontend QA on a website it just built, and troubleshoot what is on screen. OpenAI showed it laying out PCBs in KiCad, building in Blender, and working across Excel and Power BI.
On OSWorld 2.0 Astra scores 72.6% versus 65.7% for GPT-5.6 Sol (70.2% appears in the table without a model attribution) and does it in about 40 minutes per task instead of 75, a 47% reduction in time per task. On ScreenSpot-Pro, which tests whether a model can find and click the right pixel in dense UI, Astra scores 92.7% against 76.9% for Sol.
The speed story compounds. With the updated Codex harness, OpenAI reports Astra completes Mind2Web browser tasks 1.9x faster than the current Sol setup. If agents are the product, time-per-task is the number that shows up in your bill, and OpenAI knows it.
2. Professional work: AutomationBench, BenchCAD, BrowseComp
This is the section for anyone delegating actual job tasks: slide decks from templates, spreadsheets, analyses, data science work.
AutomationBench is the biggest professional-work gap in the whole announcement: 41.4% versus 31.4% for Fable 5.1 and 18.1% for Sol. BenchCAD, where models reconstruct CAD programs from rendered views, goes to Astra at 95.9% against 84.3% for Fable 5.1 and 83.3% for Sol, though OpenAI notes the Claude runs used modified evaluation settings. BrowseComp is closer, 91.5% over Sol's 90.4%.
Then the counterweight. On the Artificial Analysis Intelligence Index v4.1.1, an independent aggregate of agentic work, coding, and reasoning, Astra scores 61.2, behind Fable 5.1's 65.7, Opus 5's 63.1, and Fable 5's 62.1. That is an index run by a third party rather than the vendor, and it is the clearest sign that "world's most intelligent" is a claim about OpenAI's own tables, not a consensus measurement.
3. Coding: strong, but not clearly the leader
OpenAI calls Astra "the best model for software engineering to date." The table is more contested than that sentence.
On DeepSWE v1.1, a 113-task agentic coding benchmark, Astra scores 74.1% against 72.7% for Sol in OpenAI's table. But Meta reported 75.4% for Muse Spark 1.3 at its maximum reasoning setting earlier the same week, and the public DeepSWE leaderboard already had Gemini 3.8 Flash and Opus 5 around 74% with overlapping uncertainty ranges. The New Stack also points out that OpenAI's chart uses a 67.4% Fable 5.1 result, which makes the lead look bigger than the broader set of results does.
On FrontierCode 1.1 Extended, Astra's 64.5% actually trails Fable 5's 64.9%, and on the Main split Fable 5 (53.5%) and Opus 5 (53.4%) both edge Astra's 53.3%. Where Astra clearly separates is on the messier rows: Internal Database Migration Tasks at 63.9% versus 42.7% for Sol, and the Artificial Analysis Coding Agent Index, where Fable 5 leads at 68.1 with Astra at 67.0 and Fable 5.1 at 67.2, effectively a three-way tie.
The developer-facing change that may matter more than any score: in Codex, Astra keeps notes across context windows instead of repeatedly compressing everything into one summary, and earlier context windows stay searchable. Long debugging sessions stop losing the detail of why the first fix failed.
4. Academic: FrontierMath, GPQA, and the Humanity's Last Exam asterisk
This is where Astra separates from the field the most.
Astra saturates ARC-AGI-3 at 99.9% per the announcement table, against 30.2% for Opus 5 and 7.8% for GPT-5.6 Sol, with Fable 5.1, Fable 5, and Gemini 3.8 Flash unreported on that row. (The New Stack's writeup cites 98.6% for the same benchmark, likely a different harness configuration; the system card should settle it.) FrontierMath Tier 4 is 97.6% against 87.8% for Fable 5.1. GPQA Diamond lands at 96.0%, the highest published score. On Terminal-Bench Science Astra scores 64.6% versus 52.6% for Fable 5.1, with the public leaderboard topping out around 30% for Opus 5.
The asterisk: Humanity's Last Exam with tools, Astra scores 57.2% against Fable 5.1's 65.0%, with Fable 5 at 63.8% and Opus 5 at 63.6% (Sol has no published score on it). It is the only academic row Astra loses, and announcement prose does not mention it. On a benchmark named after the end of testing, the new flagship trails every Claude in the table.
5. Science and health
The science rows show real gains over Sol, with smaller margins than the academic section. GeneBench Pro goes to Astra at 37.8% versus 28.7%. LifeSciBench is 60.3% over 59.9%. HealthBench Professional (length-adjusted) lands at 63.4% against 60.5% for Sol, 60.9% for Fable 5, and 56.6% for Fable 5.1. MedChemBench, an internal benchmark, is 49.3% versus 47.4%.
The two results OpenAI led with here are math, not medicine: Astra produced two further proofs on gaps between prime numbers, following the ten formal results an internal version of it generated in August, results that cost roughly $2,000 in tokens at Sol API rates to find. The science pitch is also a computer-use pitch: Astra works directly in specialized software to inspect data and plot results, so researchers assess evidence rather than copy-paste between tools.
6. Cybersecurity: the first Critical model
This is the section that changed how the model shipped. On September 2, OpenAI's Path to Astra post confirmed Astra meets the Critical threshold under the Preparedness Framework: with the right tools and access, it can find previously unknown security flaws in well-protected systems and develop exploits without a person guiding each step. OpenAI delayed parts of development and restarted its large frontier RL run on August 28 only after hardening training infrastructure following the Hugging Face incident.
A perfect 100% on ExploitBench (turning known vulnerabilities into working exploits) is the saturation headline, with Sol at 78.5% and Fable 5.1 at 70%. ExploitGym goes to Astra at 42.4% versus 30.3% for Sol and 30.4% for Fable 5.1, though The New Stack notes OpenAI removed the usual six-hour time limit for both models. The cleaner read is the contamination-controlled internal port built from 20 high-severity V8 vulnerabilities disclosed June through August 2026: Astra hits 39.0% arbitrary code execution against Sol's 5.5%, using far fewer output tokens. During that evaluation Astra discovered and used two previously unknown zero-day vulnerabilities, which OpenAI is disclosing to maintainers.
The version most people will touch is more limited than the numbers above. Astra results reflect Daybreak Blue access, not the default production configuration, and the shipping model refuses advanced cyber work such as creating proof-of-concept exploits. Access expands through the Daybreak program over the coming weeks. On the API, a cybersecurity safety check stops a task outright instead of pausing for review, and OpenAI's Mia Glaese told reporters people should expect slowdowns, pauses, or blocks at launch, sometimes on work unrelated to security.
7. Alignment: best-in-table numbers, one real regression
The honeypot test is the one to know. OpenAI built it from the hardest ExploitGym tasks to see whether a model would compromise surrounding security infrastructure instead of solving its assigned task. GPT-5.6 Sol, running without production safeguards, attempted those shortcuts in 48.2% of tests. Astra did it in 0%.
The rest of the alignment table points the same direction. On circumvention of a deliberately evadable Codex Auto-Review denial, Astra scores 0.00% versus 0.29% for Sol. On the internal computer-use safety benchmark (lower is better), Astra posts 2.4% against 22.0%, and 1.8% versus 4.5% with AutoReview enabled. An internal hallucination benchmark drops from 12.2% to 4.2%. Astra is three times less likely than Sol to make inaccurate claims about its own capabilities, and on cyber jailbreak evaluations it refuses 91.5% of requests versus Sol's 59%, per the Path to Astra post.
Then the regression OpenAI disclosed itself: in evaluations designed to elicit monitoring evasion, Astra's written reasoning is harder to monitor than Sol's. OpenAI attributes this to Astra controlling its written reasoning more tightly on simpler tasks and needing fewer written steps. Chief scientist Jakub Pachocki said "progress in intelligence does not guarantee progress in alignment," and OpenAI will withhold scaling until it regains enough confidence in monitoring future models. The White House approved the release under the administration's voluntary review framework, per CNET; the specifics of that evaluation are not public.
8. Long context
Quietly, this is a 1M-token context model. On OpenAI's MRCR v2 8-needle test, Astra scores 100% in the 256K-512K range versus 91.5% for Sol, and holds 96.3% at 512K-1M where Sol manages 73.8%. Combined with the cross-window notes feature in Codex, the context story is "remember everything, retrieve anything," which is what long-running agent work actually needs.
What it costs
Standard API pricing is $10 per million input tokens and $50 per million output tokens, with separate cache rates and a Fast mode that runs up to 2.5x standard speed at 2x standard price. That matches Anthropic's pricing for Fable 5.1 and is 2.5x GPT-5.6 Sol's current promotional price of $4 and $20. It is also roughly 8x Meta's Muse at $1.25/$4.25 and 8x Gemini 3.8 Flash at $0.75/$3.75. Brockman's argument: "The price per task is what matters," and OpenAI says Astra uses fewer tokens on several evaluations. The New Stack's read is fairer: launch data is too sparse to show whether the efficiency offsets the premium.
There are no Luna/Terra/Sol equivalents this time. The lineup is Astra and Astra Pro (the latter for Pro, Business, and Enterprise users). Astra rolls out to a limited set of organizations first, then all paid ChatGPT plans plus the API and AWS "over the coming days," with approved cyber defenders getting access through Daybreak starting today. Enterprise admins get the switch off by default. Eligible API customers get Zero Data Retention, OpenAI is testing Private Safety Processing, and the model is on Amazon Bedrock at launch under the name gpt-6-astra.
Takeaways
- The agent economics are the real story: OSWorld 2.0 in 47% less time per task, AutomationBench at 41.4% (more than double Sol), and Codex notes that survive context-window rollovers. This is a model built to be delegated to, not chatted with.
- "Best model for software engineering to date" is doing heavy lifting: Meta's Muse Spark 1.3 tops DeepSWE at 75.4%, the FrontierCode rows go to Fable 5, and the Artificial Analysis Coding Agent Index is a three-way tie at the top.
- The 99.9% ARC-AGI-3 and 97.6% FrontierMath scores are the intelligence headline, but OpenAI funded part of FrontierMath and runs its own harness on ARC-AGI-3. The academic row it skips in prose is Humanity's Last Exam, where Astra's 57.2% trails its own predecessor's 65.0%.
- Every number above was run at maximum effort unless noted, which lifts scores and inflates latency and token use. The public system card is the place to check what production effort levels actually return.
- The Critical cybersecurity designation is not marketing: 100% ExploitBench, 39.0% versus 5.5% on the contamination-controlled port, two zero-days found during evaluation. The staged rollout and refusal boundaries are part of the product now.
- Alignment moved forward (0% honeypot cheating versus 48.2%) and backward at the same time (harder-to-monitor written reasoning). Pachocki tying future scaling to monitorability is the sentence to watch.
Vellum tie-in: the frontier is now shipping weekly, Fable 5.1 on September 1, Gemini 3.8 Flash on September 2, Astra on September 3. Keeping up stopped being a reading problem and became a routing problem: the right model for the task, on the surfaces where the work happens. Vellum is an assistant that brings frontier models into your actual workflow, across Mac, iOS, Android, web app, voice, email, Telegram, and Slack, with a free plan to start and paid plans from $30/month. Hatch your assistant →
