Blob-first Retrieval for Sparse Workloads

Blob storage changes retrieval from keeping every document live in RAM to keeping most of the corpus parked cheaply in object storage, which is why billion document experiments become operationally simpler. The hard part in an in memory design is not just query cost, it is the up front work of loading, sharding, and reindexing massive datasets across enough machines, plus the network cost of pushing that data into a managed system. A blob first setup turns many new retrieval jobs into data layout and ETL work instead of cluster sizing.

• A concrete example is spinning up retrieval over a huge archival corpus or many low traffic workspaces. If most namespaces are queried rarely, keeping them all hot in memory wastes RAM, while object storage lets the system leave cold data parked until needed.
  1 sacra 2 sacra
• This is why namespace heavy designs fit the model better than index per tenant systems that expect every tenant to have dedicated live capacity. The operator avoids coordinating fleets of instances for billions of documents, and the hosted system hides much of that shard and capacity planning work.
  1 sacra 4 sacra
• The tradeoff is that easy spin up does not mean easy serving for fast interactive workloads. Rare partitions can need warm starts, and once many parallel searches fan out across namespaces, the slowest cold read sets the tail latency. That pushes Turbopuffer toward archival retrieval and away from always on agent fan out.
  1 sacra 2 sacra 4 sacra

Going forward, blob first retrieval is likely to win more of the very large, sparse usage workloads where storage cost and setup simplicity matter more than tight latency guarantees. The systems that capture the most value will be the ones that pair cheap cold storage with better schema interoperability, stronger import paths from Postgres and Elasticsearch, and more predictable warm start behavior.