Inside the Checkout Bench lab
The lab is where every Checkout Bench run takes place, a signed-in workspace that sends real shopping requests to AI assistants through Nook, records everything each one did and scores the result against known answers.
Quote suite
Quote suite sends one shopping request to several AI assistants at once, and each one uses Nook to get a real checkout quote from a real, connected store account. You pick the store, the assistants and what the shopper asks, and every run stops before payment.
Run view
Every assistant in a run gets its own verdict card with the quote total or the blocker, whether the request matched, and its time, tokens and cost. Its details open onto the requests it sent, the quote or blocker, the conversation, the checks, the findings in order and the full trace.
Overview
Overview shows what the lab is doing and what it can reach, with the lab’s status, the assistants on hand, the stores connected through Nook and a fresh sandbox VM for each model, above the most recent runs.
Runs
Runs lists every run the lab kept, newest first and grouped by day, with counts of completed and failed runs and the share of clean verdicts, and each row opens, replays or downloads its evidence.
Trace replay
Trace replay opens a single run as “Inside this shopping run,” with what the assistant actually said, the requests it recorded and their outcomes, and a step-by-step timeline you can play, pause and filter by source.
Model comparison
Model comparison sends one request to separate assistants, each in its own VM with real tools and responses, and lays their results side by side so the models can be read against each other.
Bench
Bench runs fixed shopping tasks with known right answers, starting from an answer sheet, scoring every attempt as a correct or wrong checkout, and collecting the results in a scorecard with the causes behind each failure.
Accounts
Accounts lists the stores connected to the lab through Nook and when each one was last verified, next to the stores still available to connect in Nook.
Retail Profiles
Every request can run as one consistent shopper, with a short written profile that a simulated shopper answers the assistant’s questions from or that the assistant keeps as its own notes. The profile travels with the real store accounts connected through Nook, so the sizes, addresses and choices an assistant asks about come from the same person each time.