Preventing drift during index migrations

The real migration risk is not moving bytes, it is serving two truth systems at once without users seeing mismatched answers. During a rearchitecture, every upload, delete, and metadata change has to land in both the old store and the new one, and if one write lags or fails, retrieval becomes visibly inconsistent. That is especially painful in RAG and agent products, where a missing document looks like the system forgot something it knew a moment ago.

• Postgres keeps source data and retrieval close together, so updates can be fast, but teams usually add caches in front of it, and then the hard part becomes invalidating stale cache entries after every write. Moving to Turbopuffer saves infrastructure work later, but first requires a new ingestion and schema orchestration layer.
  1 sacra
• Weaviate exposes the consistency tradeoff directly. Its replicated data path is eventually consistent, with tunable read and write levels plus repair on read and async replication. That makes drift a built in systems problem, not just an application bug, especially during updates and deletes across replicas.
  3 weaviate
• Pinecone sits closer to the always on in memory model, which is simpler to reason about for live serving paths, but costly at very large scale. Turbopuffer shifts the trade toward cheaper object storage, and its own guarantees emphasize database style consistency semantics inside the system, but teams still have to keep external source data and serving behavior aligned during migration.
  1 sacra 2 sacra 4 turbopuffer

As retrieval stacks split into source database, vector index, cache, and reranker, migration discipline becomes a core product capability. The winners will be the systems that make dual writes, backfills, deletes, and cutovers feel boring, because that is what lets teams change infrastructure without breaking user trust.