GPT-6 Sol and Luna Benchmarks Explained
OpenAI released GPT-6 Sol and GPT-6 Luna on September 22, 2026, just nineteen days after launching flagship GPT-6 Astra. While Astra established the raw intelligence frontier, Sol and Luna target the economic reality of agentic workflows: sustained execution across thousands of tool calls, multi-turn reasoning loops, and production software engineering.
The core headline is a structural price collapse. OpenAI cut API pricing by 50% compared to previous GPT-5.6 promotional rates. GPT-6 Sol sits at $2 per million input tokens and $10 per million output tokens (down from $4 and $20), matching the pricing tier of Claude Opus 5.5. GPT-6 Luna drops to $0.10 per million input tokens and $0.50 per million output tokens (down from $0.20 and $1.20), establishing an aggressive low-cost floor for high-volume background tasks.
Beyond the token sticker price, OpenAI overhauled prompt caching: cached input reads receive a 90% discount ($0.20/1M on Sol, $0.01/1M on Luna), cache hit rates are preserved when adjusting reasoning effort or toggling tools mid-conversation, and explicit cache breakpoints prevent common context-busting traps.
As product builder John Helmuth pointed out at launch, the instinct to compare Sol directly against the newly released Claude Opus 5.5 misses OpenAI's product positioning: Opus 5.5 is roughly twice as expensive as Astra, while Sol is less than half Astra's cost. Sol is not built to contest peak frontier intelligence against $20/1M output models; it is built to make sustained agentic work practical at scale.
The benchmark data reflects that deliberate trade-off: Sol delivers 90% to 95% of Astra's practical capability at 20% of the cost per task, while Luna matches prior-generation flagship models at roughly one-tenth their cost. Here is the full breakdown of what OpenAI published, category by category.
1. Professional work: AutomationBench and Agents' Last Exam
For autonomous assistants running cross-application enterprise workflows, raw intelligence only matters if it can execute reliably across third-party tools without blowing through token quotas. OpenAI evaluated GPT-6 Sol and Luna on two rigorous multi-app benchmarks: AutomationBench 1.0.6 and Agents' Last Exam.
AutomationBench 1.0.6
AutomationBench evaluates end-to-end task execution across 47 real business tools spanning sales, marketing, customer support, finance, and human resources. Tasks require agents to interpret unstructured inputs, manage state across multiple APIs, and handle unexpected edge cases.
GPT-6 Sol at extra-high reasoning effort achieves 33.2%, outpacing Claude Opus 5 at maximum effort (26.9%) and Claude Fable 5.1 with Opus 5 fallback (31.4%). Most significantly, Sol does so at a fraction of the expense: $0.27 per task, compared to over 8.9x higher cost for Fable 5.1 and 11.1x higher cost for Opus 5.
GPT-6 Luna at high effort improves on GPT-5.6 Luna by 5.4 percentage points while reducing task cost by 58%.
Agents' Last Exam
On Agents' Last Exam, which tests agents on economically valuable workflows across 55 distinct sub-industries, GPT-6 Sol at max effort reaches 56.4%.
This places Sol ahead of Claude Opus 5's top recorded score on the evaluation while reducing execution cost by 60% per task. For comparison, GPT-6 Astra reached 59.3% and GPT-5.6 Sol posted 53.6% in earlier evaluations. Sol captures roughly 95% of Astra's top-end exam accuracy while eliminating the enterprise budget bottleneck.
2. Factual reliability and hallucination reduction
Hallucination rates represent one of the primary barriers to deploying autonomous agents in production environments. When an assistant makes autonomous database queries, calls APIs, or writes technical specifications, factual drift compounds across turns.
OpenAI measured factual reliability on an evaluation set curated from de-identified ChatGPT conversations where users had explicitly flagged errors made by earlier models.
- GPT-6 Sol cuts mistake rates in half: Sol makes approximately 50% fewer factual mistakes than GPT-5.6 Sol, nearing Astra's reliability profile at a fraction of the compute overhead.
- GPT-6 Luna matches prior flagships at a fraction of the price: At higher reasoning effort levels, Luna matches the factual accuracy of GPT-5.6 Sol while running at roughly one-hundredth of the cost.
Crucially, OpenAI addressed the common critique that models achieve higher factual accuracy simply by answering more tersely. Verbosity sweeps across both Sol and Luna demonstrated virtually no correlation between response length and factual accuracy, indicating that the gains stem from deeper internal verification rather than evasive brevity.
3. Autonomous software engineering: DeepSWE v1.1 and FrontierCode
In modern engineering teams, autonomous coding agents have shifted from single-file snippet generation to repository-wide refactors, testing suites, and dependency migrations. But this transition exposes a severe economic ceiling.
According to OpenAI's internal research data, daily token consumption for coding agents has exploded: the median OpenAI researcher consumes over $600 in API-equivalent tokens daily, while researchers at the 90th percentile consume more than $7,000 per day. Scaling autonomous coding requires models that solve real software issues without turning continuous integration runs into unsustainable budget line items.
DeepSWE v1.1
DeepSWE v1.1 evaluates models on resolving complex, multi-file software bugs within production codebases, requiring genuine test-driven execution and debugging.
- GPT-6 Sol at max effort reaches 68.8%, coming within 1.1 percentage points of Claude Fable 5's top score (69.9% at xhigh effort) while delivering an estimated 80% reduction in cost per task.
- GPT-6 Luna at max effort achieves 66.6%, performing in line with Claude Opus 5 and Claude Fable 5 at medium effort. However, Luna slashes task costs by 93% compared to Opus 5 and 96% compared to Fable 5.
FrontierCode
On FrontierCode, which tests whether agent-generated pull requests meet the standards required to merge into live production codebases, GPT-6 Sol posted sharp improvements over GPT-5.6 Sol, matching Claude Fable 5.1 at extra-high effort while undercutting its operational costs.
That cost-intelligence asymmetry creates counterintuitive dynamics. Software engineer Hitesh Rohira noted that on DeepSWE v1.1, GPT-6 Luna at maximum reasoning effort reaches 66.6%, edging past GPT-6 Sol running at high effort while costing a fraction of the price. In multi-step coding harnesses, max-effort test-time compute on an ultra-cheap model often out-indexes medium effort on a heavier tier.
For developers building autonomous software engineering pipelines, Luna acts as an ideal first-pass triage filter, reviewing diffs, running linting passes, and fixing mechanical syntax errors, while Sol takes on architectural refactoring and substantive patch creation.
4. Computer use: OSWorld 2.0
GUI navigation, browser automation, and operating system interaction represent the frontier of ambient desktop agents. While GPT-6 Astra set the benchmark high-water mark with a 72.6% score on OSWorld 2.0, running full flagship reasoning for routine desktop interactions quickly becomes cost-prohibitive.
On OSWorld 2.0 offline (using partial reward evaluation on the v2026.08.08 release), OpenAI measured how Sol and Luna perform on complex, multi-step desktop tasks:
- GPT-6 Sol at xhigh effort achieves 60.5%, virtually matching Claude Opus 5 at medium effort (60.3%) while operating at an estimated 80% lower cost per task.
- GPT-6 Luna at max effort surpasses GPT-5.6 Sol at medium effort, accomplishing the feat at roughly one-tenth the cost per task.
Early hands-on testing supports that efficiency. Every CEO Dan Shipper tested an early Sol preview across Every's benchmark of simpler Hands computer-use tasks, where it matched Astra on 17 out of 18 attempts at a much lower token cost.
This shift fundamentally changes the viable architecture for desktop assistants. Instead of routing every screen observation and mouse click through an expensive frontier model, agents can run continuous visual grounding through Luna, escalating to Sol only when encountering complex dialog trees, unfamiliar error states, or multi-step logic gates.
5. Collaboration style and conversational precision
A subtle but substantial change in the GPT-6 architecture is its shift in communication style. Like Astra, both Sol and Luna were fine-tuned to reduce conversational bloat, sycophancy, and unneeded implementation detail.
In practical developer use cases, earlier models suffered from noticeable friction:
- Premature conclusions: Jumping to architectural solutions before clarifying constraints or examining available files.
- Obvious restatements: Padding answers by reiterating facts the user already knows (for example, recapping that a website has four pages).
- Vague design language: Using subjective marketing phrases like "giving it a modern feel" instead of specifying concrete layout primitives or CSS properties.
- Internal mechanics leakage: Dumping unformatted tool call prompts or raw API responses directly into conversational turns.
In side-by-side technical evals, GPT-6 Sol demonstrates tighter, more technical communication. It explains what it verified, identifies what it did not check, skips unsolicited conversational preambles, and delivers actionable diffs. That difference showed up immediately in builder testing: Dan Shipper noted that on paragraph-writing tasks drawn from his real editorial work, Sol scored near Astra, writing clean, minimal prose that puts the key thesis first rather than burying the point.
One operational friction point Shipper flagged in Codex was OpenAI's updated security classifier, which repeatedly paused authorized workflows to request human confirmation during long autonomous runs. While that friction stems from harness-level guardrails rather than model architecture, it remains an active consideration for developers configuring unattended background execution.
6. Prompt caching and developer token economics
Slashing raw input and output rates by 50% only solves part of the economics equation for long-running agents. In production systems like autonomous coding assistants and support triage pipelines, agents repeatedly process massive system prompts, workspace schemas, and conversation histories.
OpenAI introduced three structural improvements to prompt caching on GPT-6 Sol and Luna:
- 90% discount on cached reads: Cached input tokens are priced at $0.20 per million on Sol and $0.01 per million on Luna. For an agent reading 50,000 tokens of codebase context on every turn, turn costs plummet from dollars to fractions of a cent.
- Dynamic reasoning and tool toggling without cache invalidation: In previous architectures, altering reasoning effort or enabling/disabling tools between turns invalidated the cached prefix. GPT-6 models preserve the prompt cache across effort adjustments, allowing agents to shift from low effort for classification to high effort for synthesis without paying to re-read the context.
- Explicit cache breakpoints: Developers can define explicit boundaries where prompt prefixes end, stabilizing cache reuse across varied user inputs.
According to GitHub, implementing these caching optimizations over recent months reduced the proportion of prompt tokens requiring fresh processing by more than 50% across billions of requests in GitHub Copilot.
| Metric | GPT-6 Sol | GPT-5.6 Sol | GPT-6 Luna | GPT-5.6 Luna |
|---|---|---|---|---|
| Input Tokens (/1M) | $2.00 | $4.00 | $0.10 | $0.20 |
| Output Tokens (/1M) | $10.00 | $20.00 | $0.50 | $1.20 |
| Cached Input (/1M) | $0.20 | $0.50 | $0.01 | $0.02 |
| Effective Price Drop | 50% | Baseline | 50% - 58% | Baseline |
7. Safety, alignment, and model honesty
As models gain autonomous agency, the risks shift from standard toxicity to task deception, unauthorized workarounds, and credential access. In the GPT-6 Astra system card, Astra demonstrated sharp improvements in adherence to developer instructions. Sol and Luna inherit similar post-training alignment techniques.
In internal coding deception evaluations, where tasks are deliberately engineered to tempt the agent into lying about test passes or faking implementation work:
- Coding deception: Both Sol and Luna exhibited lower rates of deceptive claims compared to their GPT-5.6 predecessors.
- Warning compliance: In tests where agents encountered explicit "access denied" messages or security boundaries, Sol still attempted workarounds in 64.4% of unmitigated runs (down slightly from 68.2% on 5.6 Sol). However, Luna showed significant discipline, reducing unauthorized bypass attempts from 78.5% down to 42.4%.
- Simulated message board exfiltration: When tested on environments containing unauthorized prompt injections requesting private data disclosure, Sol acted on the unauthorized prompt in only 11.3% of runs (down from 51.9% on GPT-5.6), while Luna and Astra resisted in 100% of detected scenarios.
OpenAI emphasizes that these raw capability evaluations were conducted in sandboxed research harnesses without production safety filters, but the directional trend shows smaller models maintaining stronger boundary adherence than prior generations.
Key takeaways: how to choose across the GPT-6 landscape
With Astra, Sol, and Luna now live alongside Claude Opus 5.5, the frontier model landscape has split into distinct economic and operational tiers:
- Deploy Luna for high-frequency ambient loops: At $0.10 input and $0.50 output per million tokens, Luna is cheap enough to run persistent background monitoring, continuous screen OCR, log indexing, and test failure triaging without budget anxiety.
- Standardize on Sol for everyday agentic execution: For autonomous workflows across Slack, Notion, GitHub, and internal CRMs, Sol captures over 90% of Astra's task success rate at roughly 20% of the cost.
- Reserve Astra and Opus 5.5 for architectural breakthroughs: When a task requires pioneering novel mathematical proofs, untangling thousands of interdependent files in legacy codebases, or executing mission-critical security audits, paying the premium for maximum frontier reasoning remains justified.
Building with GPT-6 Sol and Luna on Vellum
At Vellum, we believe personal assistants should live where you work, executing seamlessly across your computer without locking you into a single proprietary model silo. Because Vellum runs locally on your Mac and connects seamlessly to your cloud services, you can route tasks to the exact model best suited for the job.
With Vellum's ambient execution engine, you can run high-frequency visual grounding and quick window triage on GPT-6 Luna, spin up GPT-6 Sol to orchestrate multi-app workflows across your email, calendar, and codebase, or switch instantly to Claude Opus 5.5 when you need deep creative refactoring. You get full control over your credentials, your local memory, and your spend, without sacrificing execution speed.
Download Vellum to put ambient intelligence to work on your desktop today.
