FrontierCode Leaderboard
By The Cognition TeamFrontierCode 2.0vsCost
Score is a weighted aggregate of the rubric items. Solutions that don’t pass blocking criteria receive 0.
Cost is the mean USD spend per rollout at list API pricing, including public discounts.
Each line connects a model’s runs across reasoning efforts. The most efficient models sit top-left. The x axis uses a log scale.
FrontierCode 2.0Leaderboard
| # | Model | ||||
|---|---|---|---|---|---|
| 1 | Opus 5.5max | 45.5% | 49.7% | $12.82 | — |
| 2 | GPT-6 Astramax | 42.8% | 46.2% | $6.68 | — |
| 3 | Sonnet 5.5max | 41.0% | 45.0% | $19.94 | — |
| 4 | GPT-6.1 Solmax | 39.0% | 42.3% | $1.03 | — |
| 5 | GPT-6 Solmax | 34.3% | 37.6% | $2.35 | — |
| 6 | Fable 5.1xhigh | 32.4% | 35.8% | $8.68 | — |
| 7 | Grok 4.7xhigh | 28.6% | 32.0% | $14.36 | — |
| 8 | Haiku 5.5max | 27.7% | 30.5% | $0.37 | — |
| 9 | GPT-5.6 Solmax | 27.6% | 30.6% | $5.50 | — |
| 10 | Opus 5xhigh | 26.7% | 29.8% | $11.09 | — |
| 11 | Gemini 4 Argonhigh | 26.3% | 29.1% | $6.51 | — |
| 12 | Muse Wonder 1.0high | 24.3% | 27.4% | $3.14 | — |
| 13 | SWE-2max | 23.8% | 26.7% | $1.26 | — |
| 14 | GPT-5.5xhigh | 22.6% | 25.3% | $5.28 | — |
| 15 | GPT-6 Lunamax | 21.9% | 24.2% | $0.17 | — |
| 16 | DeepSeek V4.1 Flashmax | 19.3% | 21.8% | $0.38 | — |
| 17 | GLM 5.3max | 17.8% | 20.2% | $7.52 | — |
| 18 | Kimi K3high | 17.7% | 19.8% | $4.47 | — |
| 19 | SWE-1.7na | 16.2% | 18.9% | $3.52 | — |
| 20 | Opus 4.8max | 14.7% | 16.4% | $12.13 | — |
| 21 | GLM 5.3 Flashmax | 14.2% | 15.9% | $0.79 | — |
| 22 | Mistral Large 4high | 8.8% | 9.9% | $1.52 | — |
| 23 | Inkling Smallmax | 3.2% | 3.8% | $3.68 | — |
| 24 | Beam 1.0max | 2.9% | 3.3% | $1.30 | — |
| 25 | SWE-1.6na | 1.3% | 1.5% | $1.36 | — |
Each row shows the model’s best-scoring reasoning effort (shown in grey next to the model name). Score is a weighted aggregate of the rubric items. Click a column header to sort.
Methodology
FrontierCode is the first benchmark to measure mergeability: would the maintainer actually merge this PR?
Our criteria assess end-to-end code quality (correctness, test quality, scope discipline, style, and adherence to codebase standards) using an ensemble of grading techniques including unit tests, rubrics, and new types of verifiers.
Every task is crafted by the open-source maintainers of the repos it comes from: 20+ world-class developers built realistic, diverse, and challenging tasks, spending more than 40 hours per task, and they define what “mergeable” means in their repo. Rubric grading is subjective, so we built an extensive QC pipeline with adversarial testing, calibration, and multi-stage review, where every task is manually reviewed by a Cognition researcher.
FrontierCode also restricts internet usage. Models may use the internet the way an engineer would, reading documentation and searching for error messages, but runs that consult solution-bearing sources such as the original pull request are detected and scored zero.
Methodology revisions
Read how the benchmark is constructed, graded, and refined across revisions.
What a task looks like
Each FrontierCode task pairs a maintainer-written brief with reviewer-defined grading criteria. In this task from Mock Service Worker, a widely used library for mocking network requests in JavaScript, the model must fix a user-reported bug: resetHandlers() doesn’t restart the generator. Claude Opus 5.5 fixes the bug as described but skips the user’s finally cleanup code, and fails at every effort level. Read the full walkthrough.
Changelog
Models added to the leaderboard and changes to how results are reported.
- Results now reflect FrontierCode 2.0, the new 132-task suite. FrontierCode 1.1 and 1.0 rankings remain on their respective posts.
- Points are shaped by evaluation harness, and clicking a model group zooms into it.
- Added Kimi K3.
- Added historical results for GPT-5.5, GPT-5.6 Sol, GPT-6 Sol, Claude Opus 4.8, Claude Opus 5, SWE-1.6, and SWE-1.7, shown in the lineage animation on hover.
- Added DeepSeek V4.1 Flash.
- Added Mistral Large 4 (Preview).
- Added Gemini 4 Argon.
- Added Claude Haiku 5.5.
- Added GPT-6.1 Sol.
- Chart now supports a log scale and an output-tokens axis.
- Improved chart labels and mobile readability.
- Added Claude Sonnet 5.5.
- Added GPT-6 Sol and GPT-6 Luna.
- Added Claude Opus 5.5.
- Added Grok 4.7.
- Added SWE-2.
- Fixed GPT-6 Astra and GPT-5.6 Terra/Luna pricing, and updated GPT-5.6 Sol to promotional pricing.
- Added GPT-6 Astra.
- Added Gemini 3.8 Flash, GLM 5.3, GLM 5.3 Flash, and DeepSeek V4 Pro 0813.
- Added Claude Fable 5.1.
- Added Grok 4.6 low and medium reasoning efforts.
- Updated Gemini 3.7 Flash results to account for adjusted pricing.
- Added Gemini 3.7 Flash.
- Added Grok 4.6.
- Added Gemini 3.6 Flash, GPT-5.4-mini, Claude Opus 4.6, Claude Sonnet 4.6, DeepSeek V4 Flash 0731, Nemotron 3 Ultra, and Mistral 3.5 Medium.
- Added a flag rate column showing the fraction of runs flagged for unfair internet use.
- Updated Kimi K3 results to account for adjusted pricing.
- Updated GPT-5.6 Terra and Luna results to account for adjusted pricing.
- Added Kimi K3.
- Added Claude Opus 5 and SWE-1.6.
- Launched the leaderboard with FrontierCode 1.0 and 1.1 results.
- Corrected Claude Fable 5 costs on the FrontierCode 1.0 leaderboard.