100.00
Dimension 05 / 05
Creativity
Which model creatively generates recognizable objects?
One brief, one timed build, one photograph. The build is judged and scored out of 100.
17
Models scored
With published dimension scores
25
Scored slots per model
Excludes provisional runs
425
Recorded runs
Across models in this dimension
20%
Of the overall score
Five equally weighted dimensions
What creativity measures
Many real requests are open briefs: design a logo, lay out a room, make it look like a castle. There is no test suite, only whether the result matches what the person had in mind. Creativity measures whether a model can turn a loose description into something people recognize, choosing the scale, shape, and details that matter. Each build is timed and judged from photographs of what stands when time is up, so a model that plans too long or builds something unrecognizable scores low.
Skills tested
- 01Interpret an open-ended visual brief
- 02Plan shape, scale, and recognizable details
- 03Complete a coherent build against the clock
The Challenge: Open-brief builds
A flat world in creative mode with cheats on. The model is given one build by name and 120 seconds, and the run ends when time is up.
- Briefs
- Five builds, one run each: Godzilla, the Eiffel Tower, the Golden Gate Bridge, the Statue of Liberty, and a pelican riding a bicycle.
- Tools
- Fly, teleport, and place or fill any blocks with slash commands.
- Photo
- When time is up, the build is photographed from the front, the side and above, and the three views are judged as one image.
- Judging
- Three judges score each photograph, then compare it side by side with reference builds of the same brief. A Bradley-Terry fit of those wins and losses sets the score out of 100.
Challenge prompt
How are models prompted?Your name is Player_1 and you are a player in Minecraft.
You are a builder.
Your goal is to build what is asked for at the end of these instructions, as well and as quickly as you can, on the open grass south of spawn, centred on x=-10, z=32. You are on a flat world in creative mode with cheats enabled, so you can fly, teleport and place or fill any blocks, and you can use any blocks you like. You have 120 seconds from the start of the session, so place blocks in your very first action. The other player is a camera that does not talk: you do not need to greet it or wait for it. There is no score.
Use await cheats.command(command) to send a Minecraft Java 26.1 slash command as yourself, for example await cheats.command("/item replace entity <playerName> armor.head with minecraft:diamond_helmet 1"). Replace <playerName> with the actual player name. Keep every complete command within 256 characters including slash, spaces and NBT. Java 26.1 item stacks use count and components; entity gear uses equipment and drop_chances; text components use SNBT objects. Use /tp, /setblock, /fill, /summon, /give, /clone, /item, /clear, /enchant, /effect, /attribute, /damage, /kill, /weather, /time and /spreadplayers when they fit. /execute and /data are unavailable. Commands execute at YOUR position; @s means you, not another player. Use explicit player names or coordinates when targeting someone else's location. The tool returns whether the command was sent, not whether it succeeded. A command the server rejects comes back as a system message in your next observation; a successful command is silent. Inspect the result before claiming success. cheats.fillBlocks(blockType,x,y,z,dx,dy,dz) (inclusive deltas; zero is one block) and cheats.summonMobType also work. If a request cannot be implemented, explain the limitation rather than claim success.
When the 120 seconds are up, the camera photographs the build and the session ends.
You are in creative mode. This means: 1) you have infinite inventory and do not need to collect blocks or items, 2) you cannot get hurt, die, or lose health, and 3) you can fly: call skills.flyTo(x, y, z) to fly to any position, over obstacles or high into the air, and skills.stopFlying() to drop back to the ground.
The full participant list is Player_1, Player_2 with roles - builder: Player_1
- camera: Player_2.
The brief is not part of this prompt. It arrives as an announcement when the run starts, for example: “Build the EIFFEL TOWER. A recognisable Eiffel Tower: four splayed legs meeting in arches, two viewing platforms, and a lattice that tapers to a thin spire, up to 40 blocks tall.”
Highlights
What they are told to build
Creativity highlights: Godzilla
Drag left or right, or focus a build and use the left and right arrow keys to rotate it. Rotation loops in either direction.
Summary
A chunky Godzilla with a toothy grin, white claws and bright turquoise spines running down its back and long tail.
Summary
A broad, dark Godzilla with orange eyes, square teeth and a long tail. Gray and purple spines, some tipped in blue, rise along its back.
Summary
The plot is empty grass, with no Godzilla visible.
1 of 17: Open-brief build · Godzilla, GPT-6 Astra.
Creativity rankings
How every model performs on creativity, scored out of 100.
Updated
| Rank | Model | Score / 100 | LLM cost / run | Time / action | Valid runs |
|---|---|---|---|---|---|
| #1 | GPT-6.1 Sol | 99.76 | $0.18 | 8.1 s | 25 / 25 |
| #2 | GPT-6 Astra | 95.15 | $0.87 | 9.4 s | 25 / 25 |
| #3 | GPT-6.1 Terra | 89.54 | $0.10 | 7.2 s | 25 / 25 |
| #4 | Gemini 4 Argon | 86.98 | $0.30 | 7.9 s | 25 / 25 |
| #5 | Gemini 3.8 Flash | 79.60 | $0.086 | 9.1 s | 25 / 25 |
| #6 | Claude Opus 5.5 | 77.50 | $0.34 | 3.6 s | 25 / 25 |
| #7 | Claude Fable 5.1 | 71.17 | $0.60 | 1.9 s | 25 / 25 |
| #8 | Claude Sonnet 5.5 | 60.75 | $0.16 | 1.7 s | 25 / 25 |
| #9 | Muse Spark Open | 50.37 | $0.36 | 2.6 s | 25 / 25 |
| #10 | Muse Spark 1.3 | 42.27 | $0.23 | 3.3 s | 25 / 25 |
| #11 | DeepSeek V4.1 Flash | 40.16 | $0.040 | 2.3 s | 25 / 25 |
| #12 | GPT-6 Luna | 38.16 | $0.013 | 8.4 s | 25 / 25 |
| #13 | MiniMax M3.1 Flash | 28.06 | $0.014 | 4.4 s | 25 / 25 |
| #14 | GLM 5.3 | 18.42 | $0.011 | 26.7 s | 25 / 25 |
| #15 | Nemotron 3.5 Super VL | 12.25 | $0.014 | 4.2 s | 25 / 25 |
| #16 | Gemma 4 31B | 7.83 | $0.013 | 8.5 s | 25 / 25 |
| #17 | Grok 4.7 | 0.00 | $0.000 | – | 25 / 25 |
Ranked by this dimension only, including models with partial overall results. Costs and action times are specific to this dimension.
Inside the challenges
Explore the challenges, their rules, and how they contribute to this dimension’s score.
Challenge section 01
Creativity
Can it build what it is asked for in 120 seconds?
Open-ended (Flat World, creative mode, cheats on). The model is given one build to make and 120 seconds, and five tries at each build. A camera photographs each try from the front, the side and above, a judge panel scores the three photographs, as one image, out of 100, and the best try at each build counts.
What counts
judged build quality.
Judging applies to all runs, with 3 samples and mean aggregation. Each build is then ranked head to head against reference builds of its brief, and the score comes from that ranking.
Runs per model
- Godzilla · 5 runs
- Eiffel Tower · 5 runs
- Golden Gate Bridge · 5 runs
- Statue of Liberty · 5 runs
- Pelican riding a bicycle · 5 runs
Read the judging rubric
Read the judging rubric
---
id: creative
version: v3
kind: score
aggregateField: total
---
# Creative build quality
You are judging a Minecraft build made by an AI agent ("the builder") in creative mode within a
fixed time window. The builder was asked for one specific build by name (the "brief", given in the
facts). You see one composite image: three photographs of that one build, stacked top to bottom. The top
panel is the front view, the middle panel the side view, at a right angle to the front, and the bottom panel a
top-down view at an angle. "Front" is the camera's main view, not necessarily the way the subject faces. All three show
the build at one instant, the moment the window closed. You also see the builder's block placements and its chat. Score what was
built, not what was said.
Score out of 100 as the sum of four parts:
| Part | Max | What it measures |
|---|---|---|
| recognisability | 40 | Would a person with no caption name the brief from the three views? |
| scale_proportion | 20 | Size relative to the plain and the camera, and proportions between the parts |
| detail | 20 | Distinct features of the subject are present (limbs, spires, cables, torch, wheels...) |
| craft | 20 | Deliberate block choice, symmetry, clean edges, no stray scaffolding or debris |
Anchors for the total: **0** nothing was built; **10** an unrelated or formless shape; **50** a plausible
attempt that needs the caption; **80 and above** unmistakable at a glance.
Rules:
- Judge only the photographs for recognisability, scale and craft. Use block placements only to
resolve what a shape is, never to award points for blocks you cannot see.
- A build that is clearly in progress scores for what exists, with no credit for stated intent.
- Describe what you see before scoring, in one or two sentences.
- Judge the build as the three views together show it; a part hidden or cut off in one may show in another.
`built_something` is true if any of the three shows a deliberate structure.
- A small player figure in a panel is not part of the build.
- Confidence is `low` when the views are obstructed, too dark, or the build is out of frame in all three.
| Model | Section score / 100 | Build quality / 100 |
|---|---|---|
| GPT-6.1 Sol | 99.76 | Godzilla: 98.78 Eiffel Tower: 100.00 Golden Gate Bridge: 100.00 Statue of Liberty: 100.00 Pelican riding a bicycle: 100.00 |
| GPT-6 Astra | 95.15 | Godzilla: 100.00 Eiffel Tower: 88.01 Golden Gate Bridge: 87.72 Statue of Liberty: 100.00 Pelican riding a bicycle: 100.00 |
| GPT-6.1 Terra | 89.54 | Godzilla: 100.00 Eiffel Tower: 68.45 Golden Gate Bridge: 100.00 Statue of Liberty: 96.78 Pelican riding a bicycle: 82.47 |
| Gemini 4 Argon | 86.98 | Godzilla: 86.64 Eiffel Tower: 100.00 Golden Gate Bridge: 85.19 Statue of Liberty: 85.93 Pelican riding a bicycle: 77.14 |
| Gemini 3.8 Flash | 79.60 | Godzilla: 83.02 Eiffel Tower: 84.31 Golden Gate Bridge: 61.54 Statue of Liberty: 79.94 Pelican riding a bicycle: 89.18 |
| Claude Opus 5.5 | 77.50 | Godzilla: 62.94 Eiffel Tower: 73.30 Golden Gate Bridge: 90.25 Statue of Liberty: 70.48 Pelican riding a bicycle: 90.52 |
| Claude Fable 5.1 | 71.17 | Godzilla: 69.93 Eiffel Tower: 74.52 Golden Gate Bridge: 77.65 Statue of Liberty: 65.84 Pelican riding a bicycle: 67.91 |
| Claude Sonnet 5.5 | 60.75 | Godzilla: 57.17 Eiffel Tower: 68.45 Golden Gate Bridge: 71.42 Statue of Liberty: 54.43 Pelican riding a bicycle: 52.26 |
| Muse Spark Open | 50.37 | Godzilla: 52.56 Eiffel Tower: 37.28 Golden Gate Bridge: 67.70 Statue of Liberty: 49.87 Pelican riding a bicycle: 44.43 |
| Muse Spark 1.3 | 42.27 | Godzilla: 32.64 Eiffel Tower: 54.04 Golden Gate Bridge: 36.99 Statue of Liberty: 40.65 Pelican riding a bicycle: 47.04 |
| DeepSeek V4.1 Flash | 40.16 | Godzilla: 32.64 Eiffel Tower: 60.02 Golden Gate Bridge: 31.97 Statue of Liberty: 39.48 Pelican riding a bicycle: 36.68 |
| GPT-6 Luna | 38.16 | Godzilla: 39.75 Eiffel Tower: 18.77 Golden Gate Bridge: 51.76 Statue of Liberty: 34.79 Pelican riding a bicycle: 45.74 |
| MiniMax M3.1 Flash | 28.06 | Godzilla: 32.64 Eiffel Tower: 24.99 Golden Gate Bridge: 11.56 Statue of Liberty: 53.37 Pelican riding a bicycle: 17.74 |
| GLM 5.3 | 18.42 | Godzilla: 18.23 Eiffel Tower: 28.70 Golden Gate Bridge: 26.91 Statue of Liberty: 16.90 Pelican riding a bicycle: 1.37 |
| Nemotron 3.5 Super VL | 12.25 | Godzilla: 12.56 Eiffel Tower: 1.25 Golden Gate Bridge: 16.86 Statue of Liberty: 10.20 Pelican riding a bicycle: 20.39 |
| Gemma 4 31B | 7.83 | Godzilla: 8.52 Eiffel Tower: 8.77 Golden Gate Bridge: 3.85 Statue of Liberty: 8.46 Pelican riding a bicycle: 9.57 |
| Grok 4.7 | 0.00 | Godzilla: – Eiffel Tower: – Golden Gate Bridge: – Statue of Liberty: – Pelican riding a bicycle: – |
Data
Sample sizes, token usage, and costs behind the scores.
| Model | Recorded runs | Invalidated | Provisional | LLM calls | Failed calls | Tokens | Total LLM spend |
|---|---|---|---|---|---|---|---|
| GPT-6.1 Sol | 25 | 0 | 0 | 378 | 0 | 8,254,390 | $4.40 |
| GPT-6 Astra | 25 | 0 | 0 | 321 | 0 | 7,090,610 | $21.81 |
| GPT-6.1 Terra | 25 | 0 | 0 | 443 | 0 | 9,152,038 | $2.51 |
| Gemini 4 Argon | 25 | 0 | 0 | 250 | 6 | 5,927,170 | $7.55 |
| Gemini 3.8 Flash | 25 | 0 | 0 | 254 | 18 | 5,257,964 | $2.15 |
| Claude Opus 5.5 | 25 | 0 | 0 | 439 | 5 | 11,331,353 | $8.45 |
| Claude Fable 5.1 | 25 | 0 | 0 | 356 | 0 | 9,014,208 | $14.99 |
| Claude Sonnet 5.5 | 25 | 0 | 0 | 440 | 0 | 11,244,720 | $4.08 |
| Muse Spark Open | 25 | 0 | 0 | 458 | 3 | 11,014,904 | $9.10 |
| Muse Spark 1.3 | 25 | 0 | 0 | 436 | 1 | 8,615,241 | $5.86 |
| DeepSeek V4.1 Flash | 25 | 0 | 0 | 384 | 0 | 8,045,070 | $1.00 |
| GPT-6 Luna | 25 | 0 | 0 | 373 | 0 | 8,219,216 | $0.32 |
| MiniMax M3.1 Flash | 25 | 0 | 0 | 257 | 0 | 6,045,079 | $0.34 |
| GLM 5.3 | 25 | 0 | 0 | 107 | 0 | 1,850,105 | $0.28 |
| Nemotron 3.5 Super VL | 25 | 0 | 0 | 205 | 2 | 3,848,311 | $0.34 |
| Gemma 4 31B | 25 | 0 | 0 | 237 | 5 | 4,428,486 | $0.33 |
| Grok 4.7 | 25 | 0 | 0 | 0 | 0 | 0 | $0.000 |
Usage totals include all recorded runs in this dimension, including invalidated and provisional runs. Tokens are prompt + completion tokens; opponent costs are excluded. Provisional runs do not contribute to scores.