Search & knowledge
Keeping an AI Knowledge Base Fresh
Fresh AI answers depend on more than a scheduled import. Track source versions, indexing delays, deletions, and visible status across the knowledge pipeline.
A knowledge assistant can retrieve perfectly and still answer from obsolete material. Freshness is a property of the whole pipeline: detecting a change, fetching it, processing it, making it searchable, and invalidating dependent results. Search engines can also have a delay between writing and visibility; Elastic explains this distinction in its near real-time search documentation.
Define what fresh means for each source
Different material needs different update expectations. A product availability feed may require a much shorter delay than an internal training guide. Record a source-specific freshness target and identify the owner who can resolve connector failures. Show when the source last changed, when it was last checked, and which revision is indexed.
Use stable source identifiers so a renamed file does not become an unrelated document. Track checksums or source versions to avoid reprocessing unchanged content. Make deletion a first-class event: removing the original document should trigger removal of its chunks and any derived artifacts that should no longer be served.
Handle the awkward interval between revisions
Imagine a hypothetical logistics team updating its delivery exception procedure during the workday. Some queries may arrive while the new document is still processing. Decide whether the assistant may serve the previous approved revision, should label it as awaiting an update, or should temporarily return the source link without an answer.
Publish a new revision only when all its required chunks and metadata are ready. Otherwise a query could combine old and new sections of the same policy. Keep a generation marker that makes the active revision explicit. Avoid exposing internal processing details to ordinary users, but provide useful operational status to administrators.
Measure freshness failures directly
Track the age of the oldest pending change, failed documents, and sources that have stopped reporting. A successful scheduler run is weak evidence if individual files failed extraction. Alert on source-specific backlog or stale checkpoints, with enough context to identify the affected connector and document set.
Your rehearsal should include a document deletion, a permission change, an interrupted import, and an upstream edit made while a backfill is running. Verify that caches do not continue serving old answers after the index updates. Retain an operational audit trail so the team can determine which source revision supported an answer at a particular time.
