GamesComing soon
Equilibrium and optimal-action choices in formally specified generated games.
- Source
- Generated
- Score
- ModelAxes score, λ = 1
- Ladder
- structural · difficulty level · 1, 2, 3, 4, 5, 6
- Items
- Not built yet
- Dimensions
- stream (5)
No results yet: this benchmark is planned and is not part of any score. Everything on this page describes the plan and may change before it ships.
The plan
Draft specWritten up in a draft specification; details may change once items are built and piloted.
What it will measure
Strategic reasoning in formally specified games: finding equilibria, backward induction, eliminating dominated strategies and bargaining.
How items will be built
- Five streams: the pure equilibria of a payoff matrix, the mixed equilibrium of a small game, the subgame-perfect path through a game tree, iterated elimination of dominated strategies, and alternating-offers bargaining.
- Each stream grows over 6 rungs (matrices from 2×2 to 8×8, trees from depth 2 to 8, bargaining from 2 to 12 rounds). Games are generated so the answer is unique, and solved exactly.
- 20% of items also come as a story and as a trap: named after a famous game ("This is a Prisoner's Dilemma…") with the payoffs changed so the famous answer is wrong.
How it will be scored
- Exact answers graded by code; +1 / 0 / −1.
- x50 per stream, and for equilibrium and dominance sets, precision and recall (missing an equilibrium versus inventing one).
- How often the famous answer shows up on the trap versions.
Example (burned: never used in a scored release)
Player 1 chooses U or D; Player 2 chooses L or R. Payoffs (Player 1, Player 2): U and L gives 3, 0; U and R gives 0, 2; D and L gives 1, 3; D and R gives 2, 1. Give each player's equilibrium probabilities as exact fractions.
The game has no pure equilibrium, so the unique equilibrium is fully mixed.
Planned size900 items (5 streams × 6 rungs × 30).