Search & knowledge
Designing a RAG Architecture Around Business Knowledge
Build retrieval-augmented generation around trustworthy sources, clear document ownership, permission checks, and answers that users can actually verify.
Retrieval-augmented generation connects a language model to material retrieved for a particular request. The original RAG research combined generation with a retrievable knowledge index. For a business application, the useful architectural question is which evidence should support an answer, how that evidence stays current, and what the system should do when it cannot find enough of it.
Separate the knowledge pipeline from the conversation
Treat ingestion as a product capability with its own status, failures, and owners. A document should pass through extraction, normalization, access labeling, chunking, and indexing before becoming searchable. Store a stable source identifier and a version alongside every chunk. This makes it possible to explain where a passage came from and remove all derived content when its source is withdrawn.
The request path should authenticate the user, establish authorized sources, retrieve candidates, assemble bounded context, and generate an answer with references. Keeping these stages observable prevents a weak answer from becoming an undiagnosable model problem. A missing document and a misunderstood document need different fixes.
Design one complete question journey
Imagine a hypothetical equipment distributor building an internal assistant. A salesperson asks whether a replacement component fits a particular machine. The system needs the exact product identifier, the relevant compatibility table, and the effective document revision. A general paragraph about that product family is insufficient evidence, even if it sounds relevant.
Design the first release around that journey. Preserve table headers during ingestion, keep product identifiers searchable, and require the answer to show its source revision. If the evidence is ambiguous, return the matching documents and ask for the machine variant. This behavior is more useful than a confident guess and gives the team a measurable acceptance case.
Make operating decisions explicit
Define who can approve sources, how quickly a deletion should propagate, and which questions require human review. Establish an evaluation set containing successful queries, ambiguous wording, outdated information, and requests outside the knowledge base. Review retrieved evidence separately from generated wording so improvements address the correct stage.
Your launch checklist should include a broken connector, a revoked permission, a document replaced under the same name, and an empty search result. Also track indexing delay and source coverage. An elegant chat interface cannot compensate for a pipeline that quietly stops refreshing essential information.
