The browser cannot score itself

VSArena’s systems paper makes one claim that still feels illegal in robotics: put the scorer outside the contestant’s process, and make the failure watchable at a URL.

A robotics paper says the policy stacks blocks 74% of the time. You cannot watch the other 26%. You cannot pin the simulator build. You cannot tell whether the number came from the same commit as the code you cloned. The number is not wrong. It is simply not contestable.

That is the opening of the VSArena systems paper. It is also the reason the product exists.

Custody, not a prettier metric

Language models were stuck in the same place until evaluation moved into public arenas. What changed was not a superior loss. It was custody. The score stopped being computed by the party who benefits from it. Watching became free. Entering became cheaper than arguing. The rating became a live signal the field checked continuously instead of a table that aged in a PDF.

Embodied evaluation never made that move. Results still live where the policy runs.

VSArena copies the part that actually transferred: operator-computed scores, a public board, spectating at zero install cost, one canonical task so entries are comparable, and a submission path short enough that people will use it. Stacking has a ground truth chat never had. Either the cubes sit in their slots within tolerance, or they do not. Physics adjudicates it the same way for everyone.

The part that feels like cheating

Open a normal Chrome tab. Rapier, compiled to WebAssembly, steps a 4-DOF work-cell at 60 Hz. Three cubes. One pad. Cyan, then orange, then magenta. There is nothing to install to watch an arm drop a cube.

The policy does not get to live in that tab and grade itself. It connects to a standalone harness over a published hello → state → action → result protocol. On the public track it receives a 128×128 RGB observation and a language instruction. Cube poses stay withheld. Scoring uses them internally. The leaderboard is writable only when a trusted harness posts behind a shared secret, on a table with no client insert path. Studio demos hardcode a zero rating change. The browser cannot write the board even in principle.

That separation — browser for presence, harness for truth — is the architecture. Everything else is surface.

What a contestable result looks like

Every official result carries a provenance stamp: product version, Rapier version, physics rate, git SHA, Node version, observation mode, latency budget, scene identifier, seed, spawn hash. Failure is part of the wire format — a dotted code from a published twelve-code taxonomy, not a residual percentage. Ratings are Elo against a fixed house rating of 1200, with the continuous task-completion score used as a fractional outcome.

The paper also grades its own integrity. The hidden-scene construction is marked partial, and the held-out coordinates are readable in the repository. That is not humility theatre. It is an invitation to attack the weakest link without first having to find it.

Why one tiny task is the wow

RLBench, LIBERO, CALVIN, ManiSkill — richer suites, private custody. VSArena is poorer on task variety on purpose. If people will not run this, they will not run a bigger suite. The claim is not that Rapier replaces Isaac Sim. The claim is that embodied evaluation needed a trusted harness and a wire protocol, and both exist and run today.

The systems report is live. The empirical study across seeds and learned VLA policies is still in preparation. Watch the cell, read the board, open the paper: