AI Production Readiness Advisor

Problem worth solving

Teams ship AI features faster than they can answer a simple question: is this ready for production? Checklists live in slides; scores get hand-waved; hard constraints (ownership, evals, abuse paths) get deferred. The Advisor turns that conversation into a guided assessment with deterministic bands, hard gates, and an advisory narrative — not a certification stamp.

Intended user

Engineers, architects, and hiring audiences who want a structured production-readiness discussion for an AI feature — especially internal assistants and customer-facing copilots. The tone stays advisory and educational; it is a personal-lab showcase, not a compliance product.

Constraints that shaped the solution

• Deterministic scoring and hard gates must live in application code — never trust a client-supplied band
• LLM narrative may explain risks and remediation, but must never change scores or gates
• Answers stay in the browser; server processing is transient (cache + rate limits, no database)
• Advisory framing everywhere — no SOC2 / HIPAA / certification claims

Architecture decision

The client wizard collects context and ordinal answers across eight dimensions, persists progress locally, and posts to a narrative API. The server recomputes scores and hard gates, retrieves in-repo corpus chunks for weak dimensions, calls OpenAI for a Zod-validated narrative, and merges into a ReadinessReport. On any failure — missing key, schema miss after one repair retry, or rate limit — the user still gets scores and band.

Tradeoffs

• Accepted: keyword corpus retrieval over a vector database for MVP clarity and cost
• Rejected: letting the model adjust scores or issue certification language
• Delayed: durable KV rate limits; in-memory best-effort is honest for portfolio traffic
• Excluded: accounts, multi-tenant org workspaces, and paid certification workflows

How AI was used — and what stayed human

AI-assisted engineering accelerated the wizard, corpus, and narrative pipeline. Human judgment owned the rubric, hard-gate ceilings, trust model (server recompute), security redaction, and the advisory product boundary. The case study is judgment under constraints — not model output volume.

Privacy, security, reliability

• Assessment answers persist in localStorage only
• Free text is truncated and redacted before prompting; blocklist scrub on narrative merge
• Schema-repair retry once on invalid model JSON; then scores-only fallback
• Sample report at /sample is pre-generated — zero OpenAI cost for first-run visitors

What was delivered

• Guided assessment across eight dimensions with hard gates HG-01…HG-10
• OpenAI advisory narrative with corpus citations and degraded scores-only mode
• One-click sample report for cold visitors
• Live app at readiness.weidong-shi.com — AI in Action App #3

Lessons

1. Put band and gate authority in code before you invite a model to narrate.
2. Hard gates teach more than raw scores — ceilings make risk visible.
3. First-run trust needs a sample artifact; do not require a 10-minute wizard to see value.
4. Same series loop as RetireCheck and SleepCheck: Build → Validate → Improve → Document → Share.

Limitations and future considerations

The Advisor is not a certification, audit, or legal opinion. It does not replace your security, compliance, or risk review. Durable rate limiting and further fixture expansion can follow when evidence justifies them. The public goal is architectural credibility for AI production readiness — not a SaaS catalog.

Long-form write-up: AI in Action #3: When Readiness Needs Hard Gates.

Sibling case studies: RetireCheck · SleepCheck.