Late August: the coach learned to look again

Replay Coach v2 landed across the last week of August and the first days of September. The coach now shows you the frames it looked at, goes back for more when it is unsure, answers follow-ups with the footage in view, and grades on a scale that moves. Then every review got an assignment, and a map of the gap between what it saw and the rank you hold.

Evidence pins

Every review now keeps the frames the model actually examined, and the report pins them to the moments it talks about. A highlight at 0:42 used to be a sentence; now it is a sentence and the still it was written from. Every review since has written its frame manifest, nine of nine on the first day, so none claims to have seen something it cannot show you.

Pulling more frames

The coach can also pull extra frames. If the first pass is unsure about a moment, it asks for more stills around it before committing to a verdict. A failed pull can never fail the review: the first pass is always kept as the fallback. We skipped image caching inside that loop, against our own spec's advice: an earlier attempt measured a 0.9 percent hit rate, and this one only pays if the model pulls on roughly 28 percent of reviews, which we now count.

Chat that can see

When a follow-up question names a timestamp, the coach attaches the frame from that moment and answers with it in view. In review the timestamp parser turned out to match ordinary gaming English, phrases like at 100 HP or ping was near 50, attaching an unrelated frame and telling the model it could see something it could not. Fixed before it shipped.

A rating that moves

Over the 133 reviews completed between 1 July and 31 August, the overall rating had a mean of 5.76, a standard deviation of 1.088, and 90.98 percent landed between 5 and 7. Only seven of ten values were ever used. Three prompt bugs caused it: the schema's worked example hardcoded a 7, the rubric on the path that actually ran was one line, and a parsing default turned a missing rating into a 5. One subscriber who cancelled in August cited low quality after four reviews that all came back the same number. We de-anchored the schema, wrote one five-band rubric, and made a missing rating re-ask once instead of defaulting.

We measured the change by pairing each review's new output against its stored rating. Over 28 reviews the standard deviation went from 0.629 to 0.956, and the share inside 5 to 7 went from 100 percent to 82.1 percent. Nineteen of the 28 clips moved. That is the intended direction, and it is evidence the change bites, not a clean effect size: 28 rows against a different model configuration as well as a different prompt. Because every cross-review comparison computes a delta, ratings now carry a calibration version and never mix scales: a rescale must not fabricate improvement nobody earned.

An assignment on every review

Every review now ends with one assignment to carry into the next session, which the next review checks. This was supposed to be true already, but the instruction asking for an assignment lived in a prompt branch that had not executed since July, so the field was offered and never requested: 46 of 130 reviews came back with none. A newcomer's first review was also spending the gated daily credit instead of the signup bonus, so most first reviews rendered blurred. Both were design intent stranded on a code path that does not run; both are fixed and, as we write this, unmeasured.

The rank gap map

If you have told us your rank in a game, the report now maps each skill axis onto that game's ladder and says which rung it reads as. Your aim might read a tier above where you sit, your positioning two below. It is blunt about the skill, says nothing about the player, and promises no rank: a rating is the ladder label an axis is nearest to today, never a bar filling toward a destination tier. Almost nobody had told us a rank (15 declared against 138 ladder rows), so the report asks once, beside the radar, and never twice.