Blog · 6 min read

A datapack that installs perfectly and does nothing

The pack wrote fourteen files, loaded into a running server, enabled by name and reported success. Every one of its nine rules did nothing at all, forever, and nothing anywhere said so.

Cog can be asked for a game instead of a building: a soccer pitch where the goals count, a race with a clock, a hill worth standing on. Cog lays the arena, then writes the rules, and the rules go into the server as a datapack: a directory of .mcfunction files, dropped in, /reload confirm, /datapack enable, live. It goes in while people are playing. Nobody is kicked and nothing restarts.

The first soccer game did all of that in 1.5 seconds and then sat there being a field.

The bug

A rule that watches a region compiles to a Minecraft selector. A goal mouth is a box, so the region contributes x=…,y=…,z=…,dx=…,dy=…,dz=…, and the thing being watched for contributes type=…,tag=…. The code that built the box did not know whether the caller already had arguments, so it appended its own group:

@e[type=minecraft:snowball,tag=soccer][x=195,y=64,z=182,dx=11,dy=5,dz=4]

Two bracket groups. That is not valid selector syntax, and Minecraft's response is the interesting part. It does not complain about the selector. It refuses to register the entire function that contains it.

So the pack loaded. The pack enabled. /function soccer:init ran, created the scoreboard, laid the pitch, and printed a cheerful message in chat. And /function soccer:tick, the function every rule hangs off, answered Unknown function. Nothing in the install path asks whether the tick function exists, because why would it: it was written thirty seconds ago.

Why "it installed" is not worth saying

A build that reports the blocks it sent has not shown that it stands up. A datapack that reports the files it wrote has not shown that it plays. The only thing worth reporting is what the server does when you ask it to do the thing.

So now every rule counts its own firings:

scoreboard players add #rule_00_fired soccer 1

One line at the top of every rule, before its actions. It costs a scoreboard add per firing, and it turns every rule into something that can be asked a direct question. Before it, only rules that happened to move a score could be checked at all. The ones that only say something, like a title, a whistle or a message across the bottom of the screen, installed and were never tested. Four of nine.

Then each rule is set off on purpose. A snowball is dropped into the goal mouth. A score is driven to its winning value. The clock is wound to the tick a timer is waiting for. The counter is read before and after. Six of nine.

The last three

That left the rules that wait for a person to stand somewhere: the start pads, the finish lines, the team pads that hand out snowballs. Those were reported as untested, because a player cannot be conjured, and saying "verified" of a region nobody entered would be a lie.

Except Cog is a player. Cog joins the world like one, with feet and a position, so Cog goes and stands in them: walks to the middle of the region, waits for the tick to come round, reads the counter, and goes back to where they were. Nine of nine.

Five games were written in one evening: soccer, a shooting range, king of the hill, a sprint race and a parkour course. Between them they have 53 rules. Every one was triggered on purpose, and every one fired.

files written
14
to install, live
1.5s
rules fired on purpose
53/53

A second thing, while we were in there

The obvious way to check a pack is enabled is to ask the server and read the answer. Don't. The server truncates its own reply at around 137 characters:

There are 6 data pack(s) enabled: [vanilla (built-in)], [file/bukkit (world)], [paper (built-in)], [file/ferris (world)], [file/soccer...

The pack that is running reads as absent, because its name is cut in half. One more pack and it would drop off the list entirely. Ask the server to run the pack's own tick function instead. The answer is Running function or Unknown function, with no room for ambiguity and nothing to truncate. One extra tick of a function that already runs twenty times a second costs nothing.

A postscript: do not ask for the whole thing back

Once games could be written, the next thing anyone wants is to change one. "Make it first to three instead of five." The obvious way to do that is to hand the model the game and ask for it back with one thing different.

That does not work, and the way it fails is worth knowing. Asked exactly that, Opus 5 returned a valid soccer game with the right winning score, which had also quietly lost both team pads, one of its rules, a region, and fourteen of the thirty-seven steps that lay the pitch. Three rounds running, three different sets of losses. Every rule in the result fired, so nothing downstream had any way to notice.

Language models are bad at reproducing a long document verbatim, and the longer the game, the worse it gets. So the model no longer reproduces anything. It writes edits instead: replace rule 4, change the title, add a region. Code applies them to the original, so everything not mentioned survives by construction rather than by diligence.

[{"op": "set_rule", "index": 5, "rule": {"when": {"type": "score_at_least", "score": "red_goals", "value": 3}, ... }}]

Asked for the whole game back

  • Both team pads gone
  • One rule gone
  • One region gone
  • 14 of 37 pitch steps gone

Asked for edits

  • 4 edits, first attempt, 22 seconds
  • 9 of 9 rules kept
  • 5 of 5 regions kept
  • 37 of 37 pitch steps kept
"First to three instead of five." Left: one of the three rounds of asking for the game back. Right: the same request as edits.

Every rule was re-tested after the edit as well, because the rule nobody asked about is the one that breaks.

What the model is allowed to write

This is why the bug could be found at all. The model does not write .mcfunction files. It writes a game: teams, scores, regions, and rules of the form "when this, do that". Deterministic code turns that into commands, and every command that reaches the disk is built here, out of fields that have already been checked against a list.

So the doubled bracket was our bug, in one function, fixed once, with a test that now scans every file a game emits for ][ and fails if it finds one. A model writing raw commands would have made that mistake in a different place every time, and the fix would have been a hope.

Take a shot at the goal →

Keep reading

The other notes

Invite-only beta

A private Minecraft world for the people you choose, with Cog there to build alongside you. Join the list and we’ll email you when there’s a place.

14-day trial once invited · No card needed

A place in the first worlds

What will you
make of it?

Cubelings is an invite-only Minecraft beta. Tell us what you would build first, and we’ll email you when a place opens.

14 days when invitedNo card required

A place on the list, not a subscription. Your trial starts only when you’re invited and begin setup.

Just the details we need to invite you. How we handle your details