Aggregate score
| Rank# | Entry | Score | ±95% | $/task |
|---|---|---|---|---|
| 1 | GPT-6.1 Sol · · Medium | Score 55.4, interval ±2.2, accuracy 74.8% | 55.4 | |
| 2 | GPT-6 Sol · · Medium | Score 50.3, interval [48.3, 52.6], accuracy 72.4% | 50.3 | |
| 3 | GPT-6 Luna · · Medium | Score 26.8, interval ±2.6, accuracy 59.4% | 26.8 | |
| 4 | GPT-6 Luna · · None | Score −30.3, interval [−32.6, −27.7], accuracy 20.9% | −30.3 |
By axis
Every (model, reasoning level) entry on every axis, in aggregate order. Click a column to sort.
| 1GPT-6.1 Sol · Medium | 55.4 | 27.6 | 83.1 | $0.0052 |
| 2GPT-6 Sol · Medium | 50.3 | 19.8 | 80.8 | $0.0067 |
| 3GPT-6 Luna · Medium | 26.8 | −2.7 | 56.3 | $0.00057 |
| 4GPT-6 Luna · None | −30.3 | −17.9 | −42.6 | $0.000049 |
Knowledge
Closed-book knowledge of real entities as they get more obscure, and what a model does when it doesn't know.
| Rank# | Entry | Score | ±95% | $/task |
|---|---|---|---|---|
| 1 | GPT-6.1 Sol · · Medium | Score 27.6, interval [23.6, 31.9], accuracy 54.6% | 27.6 | |
| 2 | GPT-6 Sol · · Medium | Score 19.8, interval [15.9, 24.3], accuracy 47.6% | 19.8 | |
| 3 | GPT-6 Luna · · Medium | Score −2.7, interval ±4.5, accuracy 38.0% | −2.7 | |
| 4 | GPT-6 Luna · · None | Score −17.9, interval [−21.8, −13.6], accuracy 24.4% | −17.9 |
Math
Exact calculation and ballpark estimation without tools: how precise, and how wrong when it misses.
| Rank# | Entry | Score | ±95% | $/task |
|---|---|---|---|---|
| 1 | GPT-6.1 Sol · · Medium | Score 83.1, interval ±1.5, accuracy 95.0% | 83.1 | |
| 2 | GPT-6 Sol · · Medium | Score 80.8, interval ±1.5, accuracy 97.1% | 80.8 | |
| 3 | GPT-6 Luna · · Medium | Score 56.3, interval ±2.6, accuracy 80.7% | 56.3 | |
| 4 | GPT-6 Luna · · None | Score −42.6, interval [−45.1, −40.0], accuracy 17.5% | −42.6 |
Coming soon
Planned axes and benchmarks. They have no results yet, so they are not part of any score above.
- DiscernmentComing soon
Noticing what a question doesn't say: missing information, contradictions and assumptions that don't follow.
- Common SenseComing soon
World-state tracking, physical and social plausibility, and invariance to framing.
- StrategyComing soon
Incentive analysis, game-theoretic decisions, operational optimization, and anticipation.
- ReasoningComing soon
Constraint satisfaction, rule induction, spatial reasoning, and long-chain deduction.
- ForecastingComing soon
Inferring resolved outcomes from a fixed prior information snapshot, with calibrated probabilities.
- ResearchComing soon · v0.2
Finding and checking information on the open web: what search adds, and whether answers stay grounded.
- CodingComing soon · v0.2
Writing, fixing and reviewing code against executable tests.