Grounded RAG Chatbot — Wise Compass & My Patient Advice

Five components: a WordPress bridge, a sync worker, one Postgres with pgvector, a Fastify API that runs the retrieve → gate → generate → verify pipeline, and an embeddable chat widget. WordPress remains the source of truth; the RAG store holds embeddings, metadata, and verbatim scripture only.

Fig. 1 — System architecture. Scripture and disclaimers are enforced inside the API pipeline (stages 4–5), after the model and before the client.

Component summary

Design principles

• Grounding is structural. Scripture verbatim-ness, citations-from-retrieved-set, and the MPA disclaimer are enforced by the data layer and DTO validation — never delegated to the prompt.
• Tenant isolation below the application. Separate schemas with independent indexes; a forgotten WHERE clause cannot leak medical content into the Islamic-guidance brand.
• One store. Vectors live next to metadata in Postgres — no dual-write consistency problem, no second database to operate. Sized for 170k articles / ~350k vectors without structural change.
• Fail closed. Every uncertainty path (low retrieval score, hash mismatch, rail failure, timeout) degrades to the templated "not covered + closest articles" response — never to an ungrounded answer.

Three ingestion modes share one cleaning-and-chunking core. Re-embedding is gated on a SHA-256 of the cleaned text, because WordPress bumps modified on metadata-only saves.

Fig. 2 — Ingestion: bulk load, 15-minute incremental poll with hash gating, nightly reconciliation.

Cleaning & chunking rules

• Publish-only: status='publish' — drops MPA's 14,531 drafts from the index by construction.
• Strippers: [elementor-template] shortcodes (present in nearly every article — CTA widgets, not content), Gutenberg comment markup, spacers.
• Junk denylist: tester authors and zero-count test categories found in the real exports.
• Chunking: ~450–500 tokens, ~50 overlap, on Gutenberg block boundaries; title › h2 › h3 breadcrumb prepended; verse-reference paragraph + its blockquote always kept in the same chunk; 65k-word MPA outliers split on heading boundaries.

Postgres schema (applied to kb_wc and kb_mpa)

CREATE TABLE article (
  post_id         bigint PRIMARY KEY,
  title           text NOT NULL,
  slug            text NOT NULL,
  url             text NOT NULL,
  status          text NOT NULL CHECK (status = 'publish'),
  published_at    timestamptz,
  modified_at     timestamptz NOT NULL,
  author_id       bigint,
  reviewer_ids    bigint[] DEFAULT '{}',     -- MPA clinician credentials
  category_ids    int[]    DEFAULT '{}',
  clean_text      text NOT NULL,
  word_count      int,
  content_sha256  bytea NOT NULL,             -- re-embed gate
  last_indexed_at timestamptz
);
CREATE INDEX ON article USING btree (modified_at);
CREATE INDEX ON article USING gin (category_ids);

CREATE TABLE chunk (
  id           bigserial PRIMARY KEY,
  post_id      bigint NOT NULL REFERENCES article ON DELETE CASCADE,
  seq          int    NOT NULL,
  heading_path text,
  body         text   NOT NULL,
  token_count  int    NOT NULL,
  embedding    halfvec(1024) NOT NULL,        -- voyage-4
  tsv          tsvector GENERATED ALWAYS AS (to_tsvector('english', body)) STORED,
  UNIQUE (post_id, seq)
);
CREATE INDEX ON chunk USING hnsw (embedding halfvec_cosine_ops) WITH (m=16, ef_construction=64);
CREATE INDEX ON chunk USING gin (tsv);

CREATE TABLE scripture (
  id         bigserial PRIMARY KEY,
  post_id    bigint NOT NULL REFERENCES article ON DELETE CASCADE,
  chunk_id   bigint REFERENCES chunk,
  seq        int,
  ref_label  text,                             -- "Surah Al Tawbah (9), Verse 128" — from the PRECEDING paragraph
  quote_html text NOT NULL,                   -- byte-for-byte wp-block-quote HTML
  sha256     bytea NOT NULL
);

CREATE TABLE ingest_run (
  id bigserial PRIMARY KEY, kind text NOT NULL,  -- 'bulk' | 'poll' | 'reconcile'
  watermark timestamptz, upserted int, deleted int, re_embedded int,
  started_at timestamptz DEFAULT now(), finished_at timestamptz, error text
);

One hybrid SQL query per request — no separate narrowing layer. A calibrated answerability gate sits before the LLM: below threshold, the model is never called and the API returns the templated "not covered + closest articles" response.

Fig. 3 — Request lifecycle. Two exits before the model: the safety classifier (step 2, MPA) and the answerability gate (step 5).

The hybrid query

WITH vec AS (
  SELECT id, post_id, row_number() OVER (ORDER BY embedding <=> $query_vec) AS r
  FROM kb_mpa.chunk ORDER BY embedding <=> $query_vec LIMIT 50
),
fts AS (
  SELECT id, post_id, row_number() OVER (ORDER BY ts_rank_cd(tsv, q) DESC) AS r
  FROM kb_mpa.chunk, websearch_to_tsquery('english', $query_text) q
  WHERE tsv @@ q LIMIT 50
)
SELECT id, post_id, SUM(1.0 / (60 + r)) AS rrf_score
FROM (SELECT * FROM vec UNION ALL SELECT * FROM fts) fused
GROUP BY id, post_id ORDER BY rrf_score DESC LIMIT 20;

• Pipeline numbers: top-50 per arm → RRF fuse (k=60) → top-20 → cross-encoder rerank → 6–8 chunks, max 2–3 per article into generation. Categories are an optional explicit facet only — never an inferred pre-filter.
• "Not covered" gate: reranker-score threshold calibrated per tenant on ~200 labeled in/out-of-domain queries; a structured insufficient_context flag from the model is the second gate.
• Latency budget: embed ~80ms · hybrid query ~30ms · rerank ~120ms · generation 1.5–3s streamed → p95 < 4s, first token < 1.5s on the streaming path.

The verbatim-scripture pipeline

Fig. 4 — Verbatim scripture is a data-layer guarantee: 7,898 Wise Compass articles (90%) contain Gutenberg blockquotes; none pass through the model.

Contracts defined once as zod schemas → OpenAPI 3.1 → generated client types. Versioned at /v1, additive-only — cached WordPress embeds cannot be force-upgraded.

const ChatRequest = z.object({
  message:        z.string().min(1).max(1000),
  conversationId: z.string().uuid().optional(),   // multi-turn (phase 2)
  stream:         z.boolean().default(true),
});
// tenant resolved SERVER-SIDE from X-Site-Key + Origin — never from the body

const Segment = z.discriminatedUnion('type', [
  z.object({ type: z.literal('prose'),     text: z.string() }),
  z.object({ type: z.literal('scripture'), html: z.string(),
             reference: z.string(), articleId: z.number() }),   // server-substituted, hash-verified
]);

const Citation = z.object({
  articleId: z.number(), title: z.string(), url: z.string().url(),
  author: z.string().nullable(), reviewer: z.string().nullable(),   // MPA credentials surfaced
  score: z.number(),
});

const ChatResponse = z.object({
  requestId:       z.string().uuid(),
  status:          z.enum(['answered', 'not_covered', 'safety_redirect']),
  segments:        z.array(Segment),
  citations:       z.array(Citation),      // non-empty + from retrieved set, or API returns fallback
  closestArticles: z.array(Citation),      // populated when not_covered
  disclaimer:      z.string().nullable(),  // NON-NULLABLE for MPA — server-populated
  safetyFlags:     z.array(z.enum(['emergency','crisis','dosage_seeking'])),
  usage:           z.object({ inputTokens: z.number(), outputTokens: z.number() }),
});
// Errors: RFC 9457 application/problem+json

Streaming — typed events, tenant-specific release policy

Fig. 5 — SSE event order per tenant. Raw model deltas never reach the client; the widget shows a "checking sources" state during the MPA buffer.

• Transport: POST /v1/chat with Accept: text/event-stream; same endpoint with stream:false returns full JSON. Widget uses fetch()+ReadableStream (EventSource can't POST).
• Connection discipline: PgBouncer transaction mode — retrieve, release the DB connection, then open the Anthropic stream. 15s heartbeats, 30s hard timeout → "not covered" fallback, per-IP concurrent-stream caps, X-Accel-Buffering: no.

API surface

Token budget per query (single-turn)

Basis: ~4,550 input (1,300 cached at 0.1×) + ~450 output. Add-ons: MPA safety rails ≈ +$0.003/query on the medical tenant only; multi-turn ≈ +40% input. Three dampeners stack: prompt caching (30–50% of input spend), exact-match Redis response cache (repeat queries → $0), and the "not covered" gate (gated queries skip the LLM, ~$0.0002).

The Growth tier buys three things the Lean tier (~$31/mo, single instance) doesn't have: a redundant API pair behind a load balancer, an isolated staging environment for safe releases, and bigger DB compute to keep the HNSW index fully in RAM as the corpus grows toward 170k articles.

Fig. 6 — Growth-tier topology, ~$119/mo infrastructure. Every component EU-resident.

Environments & promotion flow (staging areas)

Fig. 7 — Three staging areas. Dev costs nothing (local + free Neon branches); staging idles to near-zero when unused; prod carries the redundancy.

Growth-tier infrastructure — line items

Development-phase AI spend (one-time, during the ~6-week build)

Expected all-in monthly — Growth tier

At Sonnet 5 intro pricing (through Aug 2026), subtract ~33% from the generation line. The per-tenant daily spend circuit breaker converts these projections into a contractual ceiling: when tripped, the tenant degrades to retrieval-only "closest articles" mode instead of overspending. Lean-tier reference for comparison: ~$31/mo infra, single instance, schemas-as-staging — appropriate until launch traffic is proven.

Every risk carries a concrete, testable defense. The four criticals are release-blocking.

Six weeks to full launch with AI-assisted development (a conventional team would quote 10–12). The compression comes from generating the ingestion pipeline, DTO layer, and eval harness against the real exports already in hand — no waiting on sample data.

Fig. 8 — Build timeline. Overlapping phases run as parallel tracks; the two amber bars are client-gated, not engineering-gated.

One fixed price for the complete, launched system — both knowledge-base assistants, live on your sites. No hourly billing, no surprises.

Fixed-price scope. Optional later enhancements — multi-turn conversation, Arabic-query tuning, high-availability — are quoted separately as Phase 2.

Payment milestones

After launch — running costs