A Chatbot That Refuses to Lie

Context

A global marketing trade association needed an AI assistant grounded in their flagship research corpus — a body of evidence their members rely on for high-stakes marketing decisions. Domain experts would use it for Q&A, so an answer that sounds right but isn’t supported by the corpus is worse than no answer at all.

The Problem

Generic chatbots hallucinate. For this audience, unsupported claims were unacceptable: the target was at least 95% grounded responses, with inline citations pointing to the exact source passages behind every claim.

My Role

End-to-end owner: UX research, product definition, architecture decisions, AI direction, and delivery — scoped as a 160-hour engagement.

Approach

The build ran as a phased product process, not a prompt experiment:

• Research before code. Documented personas, hypothesis statements, a PRD, and target metrics — the same structured prompt chain later packaged as my UX prototyping plugin.
• Citation-first architecture. Every claim traces to a source passage. The retrieval layer returns passages with stable identifiers; the response layer is constrained to cite them inline.
• A content pipeline with a written spec. Markdown to structured JSON chunking governed by a chunking guide, plus validation passes that caught chunking defects before they reached the index.
• A query router and knowledge-base introspection UI, so failures are inspectable instead of mysterious.
• Prototype → MVP split, with user testing between the two.

The discipline running through this approach is the site’s thesis: Architecture Is the Interface Now.

What I’d Do Differently

Publish the evaluation harness first. The grounding bar shaped every architecture decision, and having the measurement in place from day one would have made the early prototype iterations faster to judge.