Co-Design Bench finds top coding model completes 26.4% of tasks

A new benchmark measures how often coding agents finish a task, not just how many checks they pass.

• Claim: The best of ten tested endpoints passes 69.9% of acceptance criteria per task but fully satisfies only 26.4% of tasks. arXiv
• Evidence: 273 tasks from real coding sessions, graded against 2,331 expert-written criteria. The results table names the top endpoint as gpt-6-astra, then opus-5 (66.1% of criteria, 18.3% of tasks) and qwen3.8-max-0902 (65.3%, 16.5%). arXiv
• Caveats: The ten endpoints are not every lab’s newest model, so this is not a current leaderboard. The per-track tables were not read. arXiv
• Why it matters: Partial credit makes agents look much better than finished work does. arXiv