Oct 12, 2026 · found Nov 8, 2026, 5:55 PM PT
FrontierCode leaderboard: best agent scores 52%, leaders fail over 40% of tasks
Verdict: Confirmed · Agents and coding · Benchmarks
On real open-source tasks graded by whether a maintainer would merge, no model scores above 52% and pass rates top out at 58%.
FrontierCode grades coding agents on whether a maintainer would merge the change they make to a real open-source repository.
- Claim: No model scores above 52% and the best pass rate is 58%. Claude Opus 5.5 leads at 52%, with GPT-6.1 Sol and Claude Fable 5.1 at 49%. Artificial Analysis
- Evidence: Artificial Analysis ran 100 tasks, three attempts each, in one shared harness, with hidden tests, diff-size limits and an LLM review. Artificial Analysis
- Caveats: Cognition, which sells coding agents, created the benchmark, and the grader is an LLM (GPT-6 Luna). Artificial Analysis
- Why it matters: Cost differs widely at similar scores: about $0.81 per task for GPT-6.1 Sol against $10.15 for Opus 5.5. Artificial Analysis
Verdict history
- Nov 8, 2026, 5:55 PM PT: Confirmed. Results and method read on the Artificial Analysis page. Conflict of interest noted.