The model proposes. Something else has to decide.

Ask a language model whether two of your services write the same database table and you will get an answer. It will be fluent, it will name a file, it will name a line number, and you will have no way to tell whether any of that is real.

That is not a prompt problem. Fluent fabrication with a plausible citation is the failure mode that makes AI analysis useless in exactly the situations where it would be most valuable. ArchLens AI is my attempt to make that failure structurally impossible rather than discouraged.

One asymmetry

A proposer emits Claim objects — catalog, retrieval template, or GPT behind an API key. It has no authority.

A verifier adjudicates each claim by reading source files. It is deterministic, accepts nothing from the proposer except params, and is the only component that can cause a human being to see anything.

A claim's prose is never evidence for itself. Whether a statement is true is decided entirely by running its verifier against the corpus.

Evidence is a pointer, not a paragraph

Evidence became a pointer: file, line range, and a digest of the parent file at analysis time. The excerpt is read from disk when the finding is rendered, and never stored.

Prose that does not exist in a file cannot be produced by reading the file. A fabricated citation is not caught by a check — it has no representation.

Retrieval says where to look, not what is true

Hybrid retrieval — BM25 plus a local vector scorer, fused with reciprocal rank fusion — is scoped to one project at corpus load.

My first implementation derived claims from the retrieved chunks. Adding a single fixture file later displaced one migration out of top-k, and a true coupling silently stopped being reported.

The fix separated roles: retrieval selects the topic; the corpus supplies the evidence — the service set for each candidate is computed over every SQL chunk in the project.

A gate that can fail the build

Every claim type has an adversarial counterpart in a versioned golden dataset. The dataset runs in CI between test and build.

Cases I care most about skip retrieval and inject claims directly — fabricated table, file that does not exist, unregistered verifier, path traversal in cited evidence. All four are refused.

Hard invariants — isolation, citation resolution, groundedness — must be exactly zero, always, regardless of the baseline. Latency is recorded and deliberately not gated.

What it cost

• Every claim type needs a verifier written for it
• Cross-project analysis is impossible by construction — scoping at corpus load
• Staleness detection is built and currently unreachable until runs are persisted

Why this isn't a chat window with extra steps

A chat window cannot remember drift, prove citations, repeat answers, or enforce policy on every commit. If an answer only has to be plausible once, use the chat window. This is for answers that have to be remembered, verified, repeated, and enforced.

The demo runs on entirely synthetic data with no login and no API key — because the gate is the interesting part and the gate does not need a model to work. Seven architecture decision records are in the repository, including two designs reversed when the evidence disagreed.