Different skills.
Same court.
A model’s average only tells part of the story. These tasks test implementation, debugging, documentation, language conventions, and complete applications. Open a score to see the answer behind it.
| The challenge / what it tests | GPT-6 Astra | GPT-5.6 Sol | GPT-5.5 | GPT-5.6 Luna | GPT-5.6 Terra |
|---|---|---|---|---|---|
FizzBuzzRetired warm-up Can it get the basics right? What the model has to doGenerate the right output for multiples of 3 and 5, including their overlap. Small enough that correctness and clarity should both be easy. Read the full challenge → | GPT-6 Astra10.0/10 | GPT-5.6 Sol10.0/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Binary searchRetired warm-up Does the algorithm survive the edges? What the model has to doFind a value in a sorted list, return -1 when absent, and explain the search. Empty lists and boundary indices test precision. Read the full challenge → | GPT-6 Astra10.0/10 | GPT-5.6 Sol9.3/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Can it build an interface someone else can use? What the model has to doWrap HTTP GET and POST with authentication, useful exceptions, type hints, documentation, and a working example. Read the full challenge → | GPT-6 Astra8.7/10 | GPT-5.6 Sol8.7/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Can it explain a tool well enough to use it? What the model has to doDocument a directory comparison CLI: installation, flags, realistic examples, output formats, and how it works. Read the full challenge → | GPT-6 Astra10.0/10 | GPT-5.6 Sol9.7/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
RefactorRetired warm-up Can it improve code without changing its behavior? What the model has to doTurn a messy loop into clearly named, typed, documented Python and explain each change. Read the full challenge → | GPT-6 Astra10.0/10 | GPT-5.6 Sol9.7/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Does it test beyond the happy path? What the model has to doWrite parameterized pytest cases for a duration parser, including invalid input and edge cases with precise assertions. Read the full challenge → | GPT-6 Astra9.3/10 | GPT-5.6 Sol8.3/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Can it diagnose the actual failure? What the model has to doRepair a word-frequency function against the supplied examples, explain the bugs, and preserve frequency and alphabetical ordering. Read the full challenge → | GPT-6 Astra9.0/10 | GPT-5.6 Sol10.0/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
What happens when the network misbehaves? What the model has to doFetch URLs concurrently with per-request timeouts, retries, exponential backoff, and useful results even when requests fail. Read the full challenge → | GPT-6 Astra7.3/10 | GPT-5.6 Sol6.3/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Can it keep a complicated query correct? What the model has to doAggregate revenue by country, find each top customer with deterministic tie-breaking, and calculate shares of global revenue. Read the full challenge → | GPT-6 Astra9.3/10 | GPT-5.6 Sol10.0/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Does it know how Go developers actually test? What the model has to doUse table-driven subtests, exact map comparisons, at least six cases, and a benchmark for a word-count function. Read the full challenge → | GPT-6 Astra9.0/10 | GPT-5.6 Sol8.7/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Can it handle another language’s conventions? What the model has to doWrite organized ExUnit tests for string truncation, covering boundaries, custom suffixes, errors, and Unicode. Read the full challenge → | GPT-6 Astra9.7/10 | GPT-5.6 Sol9.3/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Can it ship a whole interactive system? What the model has to doBuild a first-person raycasting engine in one HTML file, with generated textures, perspective correction, movement, and the full supplied spec. Read the full challenge → | GPT-6 Astra9.6/10 | GPT-5.6 Sol8.7/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Do the visuals and the state agree? What the model has to doBuild a playable three-reel game in one HTML file. Animation, payouts, betting, credits, and interaction all have to work together. Read the full challenge → | GPT-6 Astra8.7/10 | GPT-5.6 Sol8.3/10 | GPT-5.5—not run | GPT-5.6 Luna—not run | GPT-5.6 Terra—not run |
Counterexample LabNew / 1.1 Can it design tests that expose broken code? What the model has to doWrite exact expectations for a tiny money-transfer ledger. Twelve events must expose as many of eight deliberately faulty implementations as possible. Read the full challenge → | GPT-6 Astra7 · 7 · 7out of 8, each run | GPT-5.6 Sol7 · 6 · 5out of 8, each run | GPT-5.56 · 6 · 5out of 8, each run | GPT-5.6 Luna7 · 5 · 6out of 8 · 1 unscored | GPT-5.6 Terra6 · 6 · 6out of 8, each run |
Original challenges show saved scores out of 10. Counterexample shows each attempt out of 8. The three warm-ups are retired; their results remain part of the frozen 13-task record.
Two tests. Two scoring systems.
How well did it do the job?
The model writes a solution to each prompt. GPT-5.6 Sol judges correctness, code quality, and documentation against that challenge’s rubric.
The task score is the mean of those three dimensions, rounded to one decimal. The model’s overall coding average summarizes all thirteen task scores. Speed is reported separately.
Task score = (correctness + quality + docs) ÷ 3
This board is a frozen historical record from Sep 25, 2026, with one saved run per task. Small gaps are weak evidence: an LLM judge can make mistakes, and Sol also judges its own answers.
FizzBuzz, binary search, and refactor are retired warm-ups. The other ten remain optional baselines; all original prompts and scores stay available.
Can it catch a wrong implementation?
The model designs tests for a tiny ledger. Each test says what to do and exactly what should happen. A correct reference implementation checks those expectations.
Then the same tests run against eight faulty ledgers. Each distinct faulty version exposed by a valid trace earns one point. Code computes the result; there is no LLM judge.
Result = distinct faulty versions exposed / 8
Version 1.1 limits each submission to twelve events across at most four traces. A trace with one wrong expectation earns no coverage. Malformed or failed calls are unscored.
The first calibration used three independent attempts per model at low reasoning effort. We show every attempt. These are preliminary results on a narrow test.
The coding averages and fault counts measure different things. They stay separate on every model’s profile. Changing the Counterexample contract, faults, budget, or scoring will create version 1.2.