# How the audit works

One audit run is twelve stages. Each stage is deterministic in what it does with its
input; the non-determinism lives entirely in the two places that call a language model
(the engine panel and the judge), which is why the score has a
[measurable noise floor](reliability.md).

## Stages, in order

| # | Stage | What happens |
|---|---|---|
| 0 | **Brand upsert** | Brand row created or matched; run row opened with status `running`. |
| 1 | **Engine resolution** | Which model families are configured and reachable. Zero available engines is a hard failure — never a 100/100. |
| 2 | **Site scan** | The brand's site is crawled. Runs *before* the model panel, because the questions cannot be tailored until the site has been read. |
| 2a | **Archetype inference** | The site is classified as `ecommerce`, `saas`, `service`, or `unknown` from positive crawl evidence only. |
| 2b | **Content coverage** | Which page roles the site has and lacks, scored separately from the total. |
| 3 | **Brand profile extraction** | An LLM reads the crawled page text and extracts what the brand actually sells. |
| 4 | **Query building** | Five shopper questions built from that profile. Falls back to a generic set if extraction fails. |
| 5 | **Engine panel** | Each question is asked of four model families in parallel. |
| 5a | **Evidence guard** | Zero usable observations throws "insufficient evidence" rather than scoring 0/100. |
| 6 | **Persist** | Raw responses and site signals stored. |
| 7 | **LLM judge** | Each response is read and classified. Unjudgeable responses are dropped, not counted as invisible. |
| 8 | **Scoring** | Deterministic arithmetic, no model call. |
| 9 | **Fix generation** | Every penalty and coverage gap becomes a ranked fix. |
| 10 | **Peer comparison** | Offline, against a committed cohort snapshot. No network call. |
| 11 | **Report render** | Markdown report, optionally rendered to a branded client document. |
| 12 | **Complete** | Run marked `complete`, or `error` with the failure recorded. |

## Two design rules worth knowing up front

**Absence of evidence is never scored as evidence of absence.** If a signal could not be
verified — the site was unreachable, the crawl never reached a product page — that signal
is excluded from both the penalty and the denominator it would otherwise sit in. A site
that blocks the crawler does not get a bad score; it gets a withheld one.

**A question nobody answered is not a question you failed.** A query with zero
observations is marked `unobserved` and carries no penalty. Only a query that engines
*did* answer, and answered without you, costs points.

## Per-stage detail

[!ref Target=_self text="Discovery and crawl"](discovery-and-crawl.md)
[!ref Target=_self text="Brand profile extraction"](brand-profile.md)
[!ref Target=_self text="Query generation"](query-generation.md)
[!ref Target=_self text="The engine panel"](engines.md)
[!ref Target=_self text="The judge"](judging.md)
[!ref Target=_self text="Scoring"](scoring.md)
[!ref Target=_self text="Content coverage"](coverage.md)
[!ref Target=_self text="Measurement reliability"](reliability.md)
[!ref Target=_self text="Reports"](reports.md)