How we evaluate

1. The same prompt, the first answer

Every model gets the identical English prompt – the task plus shared rules – with no examples and no chance to try or fix the result. The answer is one self-contained HTML file, published unedited. If an output does not run, it is published as is and scores 0; an optional fix by the same model is a separate, labelled copy.

2. An 8-requirement checklist

Each prompt has 8 items taken straight from the task. Each item is ✓ met (1 point), ½ partly (0.5) or ✗ missing (0); the maximum is 8. The code and a screenshot are judged, not gameplay; every verdict points to specific places in the code.

3. Two independent AI reviewers

The checklist is filled in independently by two AI reviewers – Claude Opus 5.5 and Codex (GPT-6 Astra, medium). Where they disagree, the stricter (lower) verdict is proposed. Their agreement is shown together with the scores on every comparison.

4. Objective tests

What can be measured is measured by a tool. The perft chess test (exact node counts on standard positions) replaces the reviewers' verdict on the perft item. Technical health (Lighthouse, rendering smoothness, errors after automated clicking and keyboard use, the mobile layout and a scripted check that the output can really be used) and UX signals (control size, layout at three widths, keyboard, feedback, readability) have their own 0–100 scores. The look (UI) is judged by AI with one rubric on screenshots of the same moment (readability, detail, variety, consistent style, visual glitches) – two independent reviewers, the stricter verdict is published after human approval; on the site it stands next to UX as “UX / UI” and the order follows their sum. The headline “Overall” score (0–100) combines 60 % requirements met, 30 % UX / UI and 10 % technical health; a broken output scores 0.

5. Human approval

The AI reviewers' scores and verdicts are a draft. They appear on the site only after a human has checked and approved them. Until then the pages show only the objective measurements.

6. What the evaluation does not measure

The checklist does not measure beauty or the experience of use – that is what the arena is for, where visitors blindly pick the better of two outputs. Cost is the token count priced with the API price list as of the date shown.

So far 135 outputs from 27 models for 5 prompts. The second reviewer has assessed 134 outputs; it agreed with the first on 76 % of the checklist items.

Evaluation per prompt

• Solar System – An interactive 3D Solar System with real orbital data, time controls and a camera that flies to the planets.
• GTA-style game – A playable open-world city game in the style of the first GTA: cars, pedestrians, police with wanted stars, missions and a minimap.
• Monitor test – A professional monitor test: dead pixels, gradients, black and white level, gamma, sharpness and backlight uniformity.
• New York & Central Park map – An interactive illustrated map of Manhattan with a detailed Central Park, 30+ landmarks, search and a guided tour.
• Chess with an engine – Complete chess with every rule and its own chess engine; the move generator is verified with an objective perft test.