Turbopuffer minimizes object storage roundtrips

This is really a claim about why turbopuffer can put its index on cheap object storage without turning every query into a long chain of waits. HNSW search walks a neighbor graph step by step, so each step can trigger another dependent fetch. On object storage, each fetch can take around 100ms, so ten to twenty serialized fetches quickly blow up latency. SPFresh turns that into a small number of bulk reads, first loading centroids, then fetching the relevant clusters.

• The practical difference is access pattern. HNSW and other graph indexes depend on many random graph hops, while turbopuffer says centroid based SPFresh bounds cold query work to roughly the height of the tree, typically two to four object storage round trips, or even two in earlier descriptions.
  1 turbopuffer 2 turbopuffer 3 turbopuffer 4 turbopuffer
• That matters because turbopuffer separates compute from storage. The durable copy lives in object storage, recently used namespaces sit on NVMe, and hot ones stay in memory. Fewer dependent reads let a cold namespace still answer queries without preloading the whole index into RAM, which is the core economic advantage versus always in memory systems.
  1 turbopuffer 2 turbopuffer 5 sacra
• The tradeoff shows up at the tail. Internal interviews describe turbopuffer as strongest for very large, unevenly accessed corpora, but note that cold fetches can still hurt p90 and p95 latency in fan out workloads. That is why the architecture fits archival and generic retrieval better than highly personalized ranking systems.
  5 sacra 6 sacra

Going forward, the winning vector backends are likely to split into two camps, RAM heavy systems optimized for tight latency guarantees, and object storage native systems optimized for scale and cost. turbopuffer is betting that better clustering, caching, and hybrid retrieval can keep moving more search workloads into the second category.