Ask what the user sees before you auditAsk what the user sees before you audit

In July I told my orchestrator that the chat page on my dashboard felt very glitchy when I opened several chats. That was the whole report.

It did what a careful engineer does. It read the code. Then it sent Codex and Grok to read the code. Five rounds across three models.

Twelve real bugs

They found twelve. Not style notes. Real bugs:

• ghost sessions that saved replies under a session the tab bar never pointed at,
• duplicate transcript writes from streams that had already been replaced,
• a refresh poll every 250 ms that never stopped,
• panels mounted twice,
• a dead event bus, where one command showed a success toast and did nothing.

All worth fixing. None of them was my symptom.

One sentence

Then the orchestrator asked what I actually saw. I said: when I open a chat, I see it scrolling to the present instead of seeing the present immediately.

That sentence cracked it. One scrollTo call used behavior: "smooth" and fired on every change to the message list. Opening a chat animated from the top down, and every streamed token interrupted the previous animation. The fix was one word.

Five rounds of analysis could not find it, because nothing in that code was wrong in the way analysis looks for. It did exactly what it said. What it said felt broken.

Why audits miss the reported bug

A code audit answers the question “what is wrong here”. The user asked a different question: “why does this feel wrong to me”. The two overlap less than you would think.

Static analysis is good at invariants: races, leaks, dead paths. A user reports experience: flicker, lag, the wrong thing on screen. The bridge between the two is the observable symptom, and only the user has it.

There is a trap in the middle. Analysis always finds something, and when it finds a real bug it is very tempting to call the report solved. One of the fixes from that same week, unmounting inactive tabs, made every tab switch remount, refetch and flash. The fix was the regression. When someone says it is still broken right after a deploy, suspect the deploy.

The same mistake with numbers

Measurements have their own version of it. A hover probe reported 190 to 202 ms against a 180 ms limit. With the limit tightened to 100 ms, it reported 115 to 123 ms. The readings clustered, so they looked like signal. Two fix tasks were ordered, and one regression was escalated to me.

With the limit at 4000 ms the probe reported 4015.6 ms. It had measured its own timeout every time. Nothing was slow. The probe had never seen the event it waited for.

Same lesson. Check the instrument before you blame the product.

How I work now

• For a vague UI report, get what the person sees, in their own words, before anyone reads code. Offer concrete options: duplicated data, flicker, lost input, broken streaming.
• Ask about the axis too. In-app tabs or separate browser windows? One answer can kill a whole hypothesis.
• Run the audit anyway, in parallel. It finds real bugs. Just do not call the report fixed because it found one.
• For any performance number from a probe, raise the limit and run it again. If the number follows the limit, it is the limit.