# P13 — Ongoing QA Operations

**Updated:** 30 August 2026 · There is no automated test suite because there is no code. Quality here = calls handled well, verified on a schedule. Three routines, all founder-run, all time-boxed because their hours count against the kill line (**>10h/client/mo**).

---

## 1. What each routine proves

| Routine | Cadence | Time box | Proves |
|---|---|---|---|
| 20-call QA script (`TEST_PLAN.md`) | Pre-launch, post-major-change, quarterly | ~4h full / ~1h subset | The config handles the known scenario space |
| Weekly sampled-call review (§2) | Weekly, batched across all clients | ≤ 2h total (3 clients) | Real callers are being served well; drift is caught within a week |
| Monthly metrics & acceptance (§3) | Monthly per client | ~30m/client | Unit economics hold; the client would agree it's working |

## 2. Weekly sampled-call review (the core habit)

One batch session per week, all clients together (task `C-20`):

1. Open each client's platform dashboard → last 7 days of calls.
2. Read/lsten to a sample: **all** escalations, **all** calls flagged failed/abandoned, **all** calls > 6 minutes (length = confusion smell), + 3 random routine calls per client.
3. Score each sampled call: outcome achieved? · knowledge answer correct? · tone acceptable? · should a human have been pulled in sooner?
4. Output per client = a fix list of ≤ 3 items, applied the same session where they are prompt/knowledge edits (bigger items → `C-22`).
5. Log one line per client in `PROGRESS.md`: date, calls sampled, containment %, fixes applied, minutes on the task.

**Week-1 protocol for a fresh client (`C-16`):** same review but DAILY for the first 5 business days, every call not just a sample. Most template gaps surface in the first 50 real calls; catching them in week 1 is what makes the 2-week delivery promise honest.

## 3. Metrics from platform dashboards (monthly, per client — task `C-21`)

| Metric | Source | Healthy | Action threshold |
|---|---|---|---|
| Voice-to-voice latency (median) | Platform latency dashboard | ≤ 1.0s | > 1.2s two weeks running → config review (model/voice choice), fallback platform test if unresolved |
| Interruption/barge-in complaints in transcripts | Transcript scan ("stop talking", "let me finish") | ~0 | Any cluster → endpointing retune + case-4 retest |
| Containment (handled without human) | Dashboard + review notes | ≥ 70% of routine calls | Below → knowledge-pack gaps list |
| Escalation correctness | Weekly review | 100% of D8 triggers escalate | ANY missed trigger → immediate fix + cases 5–8 rerun; this is the one zero-tolerance metric |
| Minutes used vs tier cap | Usage dashboard | ≤ cap | ≥ cap → upsell script (`SALES_PLAYBOOK.md` §6) |
| Infra cost vs 40% floor | Usage × rates (`PRICING.md` §3) | ≤ 40% of fee | 2 consecutive breaches → re-tier |
| Bookings + leads captured | Calendar + lead log | Trending with call volume | Falling while calls steady → booking-flow QA subset |
| Our hours this client this month | `PROGRESS.md` log | ≤ 2h steady state | > 10h = **kill-criterion breach** — escalate to founder decision per `ROADMAP.md` tripwires |

## 4. Per-client acceptance checklist (monthly, sent to client as the "wins" summary)

☐ All week's summaries were delivered on schedule · ☐ No unresolved complaint about the assistant from the client or a caller · ☐ Containment ≥ 70% · ☐ Zero missed escalation triggers · ☐ Bookings verified landing in calendar (spot-check 3) · ☐ Minutes/cost inside tier + floor · ☐ Client asked "keep, change, or cancel?" and the answer is recorded in `PROGRESS.md` (this rolling answer is the early-warning system for the P8 retention gate).

## 5. Honesty rules

Failures found in our own configuration are reported to the client and in `PROGRESS.md` plainly and prominently, never buried (working-agreement rule 3). A week with no QA session is logged as "QA MISSED" — not backfilled.