How the audit works

One audit run is twelve stages. Each stage is deterministic in what it does with its input; the non-determinism lives entirely in the two places that call a language model (the engine panel and the judge), which is why the score has a measurable noise floor.

Stages, in order

#	Stage	What happens
0	Brand upsert	Brand row created or matched; run row opened with status running.
1	Engine resolution	Which model families are configured and reachable. Zero available engines is a hard failure — never a 100/100.
2	Site scan	The brand's site is crawled. Runs before the model panel, because the questions cannot be tailored until the site has been read.
2a	Archetype inference	The site is classified as ecommerce, saas, service, or unknown from positive crawl evidence only.
2b	Content coverage	Which page roles the site has and lacks, scored separately from the total.
3	Brand profile extraction	An LLM reads the crawled page text and extracts what the brand actually sells.
4	Query building	Five shopper questions built from that profile. Falls back to a generic set if extraction fails.
5	Engine panel	Each question is asked of four model families in parallel.
5a	Evidence guard	Zero usable observations throws "insufficient evidence" rather than scoring 0/100.
6	Persist	Raw responses and site signals stored.
7	LLM judge	Each response is read and classified. Unjudgeable responses are dropped, not counted as invisible.
8	Scoring	Deterministic arithmetic, no model call.
9	Fix generation	Every penalty and coverage gap becomes a ranked fix.
10	Peer comparison	Offline, against a committed cohort snapshot. No network call.
11	Report render	Markdown report, optionally rendered to a branded client document.
12	Complete	Run marked complete, or error with the failure recorded.

Two design rules worth knowing up front

Absence of evidence is never scored as evidence of absence. If a signal could not be verified — the site was unreachable, the crawl never reached a product page — that signal is excluded from both the penalty and the denominator it would otherwise sit in. A site that blocks the crawler does not get a bad score; it gets a withheld one.

A question nobody answered is not a question you failed. A query with zero observations is marked unobserved and carries no penalty. Only a query that engines did answer, and answered without you, costs points.

Per-stage detail