The runs behind every number

How to read the labels

Every figure on the site carries one or more of these five labels. Hover or focus a label anywhere on the site for its one-line definition.

Measured

A committed check result: how many of a model's answers differed from the answers recorded in the history, always with the date of the run.

Inferred

Anything an AI model wrote: a suggested answer, how sure it says it is, a hint, a flag's wording, the layout of a sample inbox.

Illustrative

A staged or drawn example, such as a sample signature or a film's title card, not a real event.

Declared

A name written down in the record, such as who signed an answer. It is not proven by a key yet.

Fictional

A made-up company or person: Larkspur Garden Co., the Hollis lab, and the people named in their records.

The sample company's check (2026-09-23)

Larkspur Garden Co. is a made-up garden-centre chain. Its 60 past invoices each carry a recorded answer, made up with the rest of the data: paid, sent to the buyer for a price check, or held for one of three reasons. Claude Sonnet 5 judged all 60 with no examples, at low effort, and its answers were counted against the recorded ones. This is the record behind the sample inbox.

Bank-detail changes and possible duplicates have only 5 past cases each, too few to lean on, so they stay with a person whatever the counts say. In the sample inbox the steward has let the AI finish nothing alone yet.

Day one: on the first imported record, before the steward edited any words, Claude Sonnet 5 missed 3 of 60 (5.0%), right at the limit of 5%. All 3 were invoices the history calls paid.

Source: Measured 2026-09-22 · the same 60 invoices with the importer's placeholder words · Claude Sonnet 5 · made-up company

The earlier record, and the check that failed (2026-09-22)

Before the sample inbox's record there was an earlier record for the same made-up company: the same kind of 60 invoices, with different answer names. Its 60 answers were written by the company's written rule (the payables policy as code), not by people. Three engines were checked against it, one run each.

The stand-in. It answers pay for every invoice, so it misses every case whose recorded answer is anything else: 34 of 60. It is a floor, not a model, and it is the failing example on the front page.

JEV 1.13.0, TypeSafe's model. On the Larkspur names it disagreed with 5 of 60 past answers (8.3%), over the same limit of 5%. FAIL. Those 60 answers were written by the company's written rule, not by people. Three disagreements were the rule's mistake: it called biweekly service bills duplicates, and the model said pay. Two were the model's: invoices within the price tolerance that it sent for a price check. On one biweekly bill the model said pay at 0.98, over that record's 0.97 bar, so the thresholds would have let the machine pay against the written rule with no person. The fix was a change to the thresholds, not a new model.

Nothing but vendor names and item descriptions changed between the 7 and the 5. Treat any single run's exact count as one sample.

On a later record with the steward's words, JEV alone missed 1 of 60 (2026-09-23; see cost and speed). platatlas.com runs one engine today, TypeSafe.

The steward's words (2026-09-23 and 2026-10-02)

The same 60 Larkspur invoices, the same models. Only the words changed. On the left, the importer's placeholders ("Recorded as Paid in 29 of 60 past cases"). On the right, each answer written the way a steward would write it, from the company's payables policy. No case, answer or threshold changed.

Measured 2026-09-23 and 2026-10-02Fictional company

Clef and Opus are not on platatlas.com; Clef's price was not measured. platatlas.com runs one engine today, TypeSafe.

On the Opus 5.5 run with the steward's words, 50 of 60 past cases would have closed with no person, and 0 of those were wrong. Bank-detail changes and possible duplicates stayed with a person.

The steward's words were written from the same policy that generated the data, which is the best case. A real steward's first draft will be rougher, and the weekly check is how it improves.

Source: Measured 2026-09-23 (Claude) and 2026-10-02 (Clef) · 60 past invoices · one run each · made-up company

Words drafted from the history (2026-09-24)

The start page drafts each answer's words from rules it counts in a team's own history, with no AI. To see how far that gets, JEV judged the same 60 past decisions of two made-up organizations four times, changing only the words: the Larkspur invoices and the Hollis lab's robot detections.

Measured 2026-09-24Fictional organizations

Run to run, the same setup varied by one or two cases: the placeholder run on accounts payable scored 15, then 13.

Drafts from a team's own history cut the AI's mistakes by about half before anyone writes a word. A steward's own words take it the rest of the way. That is why the start page drafts them for you, and why the first job we give a steward is to replace them.

We first measured the drafts on the same rows they were drafted from, and they looked perfect. Held out, on cases they had never seen, the first version did worse than the placeholders on accounts payable, because a small history lets rules fit coincidences (an account number read as a quantity; the everyday answer described by two unrelated conditions). We fixed three things: no number thresholds on identifiers, the everyday answer stays the default, and rare answers keep plain words. Every number in the table is after the fix. The start page's AI check now drafts from one half of a history and grades on the other.

GLiNER2, a small open model, untrained, on a Raspberry Pi, with the importer's placeholder words on accounts payable: 39 of 60 wrong (65.0%), FAIL. It was not run with the drafted or the steward's words. Trained on a team's own decisions it does far better; see the lab.

Source: Measured 2026-09-24 · 60 past decisions each, two made-up organizations · JEV and GLiNER2

Cost and speed, self-hosted (2026-09-23)

The Larkspur record with the steward's words, run on our own machine, not on platatlas.com. Each engine was measured once on the same 60 invoices with its cost and time for each case; the combination was scored from those runs with no further model calls.

Measured 2026-09-23Fictional company

Sure and wrong: why a probability is not a grade.

Gemma 4 e2b gave 45 of its answers at 0.99 or above, and 16 of those were wrong. A model's own sureness is inferred; only the count of its misses against the recorded answers tells you whether to lean on it.

Source: Measured 2026-09-23 · 60 past invoices with the steward's words · one run each · made-up company

Finding mistakes in your own history (2026-09-24)

We planted wrong decisions at random in each made-up history (20 repeats each) and counted how many the start page's "worth a second look" list found. Counting only, no AI.

Fictional organizations

Some flags point at decisions that were right but simply stand out; the lab has 3 of those with no mistakes planted at all. That is the trade we chose for a list a person reads in a minute: each flag says why it stands out, and a person looks.

An earlier version that flagged only exceptions to rules that hold at 90% or better had no false alarms, but found only 9 to 25% of the mistakes.

Source: Measured 2026-09-24 · 20 seeded runs for each count · two made-up organizations

A research lab, not an AP team (2026-09-23)

The Hollis Home Robotics Lab is made up: a home robot's object detections in volunteer households, each labeled by the lab's annotators. Each case is what the robot's perception had (its detector's top two guesses, a camera caption, size, room, surface and so on), with no images. The history oversamples what the detector gets wrong, as a lab's labeling queue does. 60 past labels, nine answers, limit 5%.

Measured 2026-09-23Fictional lab

This history is synthetic and its labels follow clean rules from the fields, which a small model learns easily. A real lab's detections may score lower; a check on the lab's own history is what decides. Watch the lab's film.

Source: Measured 2026-09-23 · 60 past labels · one run each · made-up lab