Thin Middle Layer in AI Retrieval
Product manager at Cohere on enterprise AI search infrastructure and deep research agents
The biggest non platform risk for Exa, Parallel, and Tavily is that much of their value sits in a thin middle layer between raw web access and the application, which makes margins and retention fragile. Cohere’s workflow shows why. Public web retrieval can be bought as an API, but the hard enterprise work is wiring messy internal systems like SharePoint, customer by customer, with custom parsing, permissions, and evaluation that do not package cleanly into a self serve product.
-
Low switching costs are already visible in practice. Cohere moved from Brave to Tavily mainly because Tavily returned cleaner text for LLM grounding. Ecosia evaluated Exa, Tavily, and Parallel and kept its architecture flexible because quality had largely converged, which suggests buyers can swap vendors if price or service changes.
-
Unit economics can get squeezed from both sides. These APIs charge per search, page read, extract, or task, so usage scales directly with customer activity. That works for enterprise deployments where each customer brings its own API key, but it becomes painful in mass market SaaS where millions of end user queries can turn retrieval into a large pass through cost center.
-
A structural vulnerability is access dependence. Parallel’s own positioning leans on crawling and monitoring the web, and Bright Data’s Unblocker category exists because many sites actively resist bots. If publishers tighten blocks, require licenses, or reduce crawlability, search quality and coverage can degrade for everyone in this layer at once.
The next durable winners will move either upward into full research workflows with planning, citations, and vertical datasets, or downward into hard to replicate data access and enterprise connectors. The middle ground, generic web retrieval sold as an interchangeable API, should keep growing near term, but it is the least defensible place in the stack as the market matures.