Benchmark desk
Scores appear in the same table only when their comparability keys match exactly. Differing harnesses, shots, or scoring methods get separate panels — never a blended ranking.
| Benchmark | Family | Rows | Contamination |
|---|---|---|---|
| Aider polyglot | aider | 9 | medium |
| FrontierMath Tier 4 | frontiermath | 11 | low |
| FrontierMath | frontiermath | 13 | low |
| GPQA Diamond | gpqa | 45 | low |
| MATH Level 5 | math | 12 | medium |
| OTIS Mock AIME 2024-2025 | aime | 44 | low |
| SimpleQA Verified | simpleqa | 30 | medium |
| SWE-bench Verified | swe-bench | 10 | medium |
| Terminal-Bench | terminal-bench | 38 | medium |
See methodology for what we refuse to compare, and sources for license tiers.