Search & knowledge
Planning an Embedding Migration Without Search Regressions
An embedding change is a search migration. Plan a parallel index, consistent query encoding, representative evaluations, and a practical route back.
Changing an embedding model can affect the representation of every searchable passage and every query. It should be treated as a data migration with a measurable search outcome. Elastic warns that mixing embedding models across indices can produce unexpected rankings. A successful migration therefore needs consistent configuration, explicit versioning, and a rollback plan before production traffic moves.
Inventory the current representation
Record the embedding configuration, dimensions, normalization assumptions, chunking version, and searchable metadata. Document which services create document vectors and which create query vectors. A background worker left on an old configuration can create a mixed corpus long after the application has been updated.
Assign an index generation identifier that includes the relevant processing versions. Keep source text and source versions available for reprocessing. This also helps estimate the work: count eligible chunks, identify documents that can be skipped, and account for the time required to catch up with edits arriving during the migration.
Run old and new paths side by side
Imagine a hypothetical service desk whose search finds both everyday language and exact error codes. Build a candidate index from the same source snapshot as the current index. Send a representative set of saved questions through both paths, ensuring each uses the matching query encoder. Compare the evidence retrieved for each question before comparing the final generated answer.
Include uncommon terminology, short queries, long questions, and documents with recent edits. A new configuration that improves broad semantic questions might weaken exact identifier lookup. Check those slices independently. For a limited rollout, log the index generation with every request so support reports can be tied to the actual search path.
Plan the cutover as an operation
During backfill, continue recording source changes so both indices can reach an agreed freshness point. Validate document counts and failed ingestion records, but do not mistake matching counts for matching quality. Verify deletions and permission metadata as carefully as successful additions.
Route requests through an index alias or equivalent application setting that can switch without a new client release. Keep the previous index available for a defined observation period, subject to retention and deletion requirements. Rehearse switching back, including the query encoder and related cache namespace. A rollback that restores only the vector store can leave the system inconsistent.
