Product manager at Cohere on enterprise AI search infrastructure and deep research agents

We spoke with a product manager at Cohere who works on North, their enterprise AI assistant for knowledge workers, and who personally uses Parallel through Manus for deep research.

Our conversation explored how search infrastructure for AI agents compares across providers like Parallel, Exa, and Tavily, examining their differentiation, pricing models, and the value of domain-specific knowledge bases for complex research tasks.

Key points via Sacra AI:

To start, tell me about Cohere—what the company does and what your role is?

I work for a company called Cohere. Cohere is a frontier AI lab. It builds LLMs from scratch as well as the software layers on top of that. I'm a staff product manager. I've been there for over 3 years. Bounced around a lot, wore a lot of different hats. But right now, I work on North. Think about North like ChatGPT Enterprise or Glean. It's our application for enterprise AI assistance for knowledge workers. Think anybody white collar that uses AI for work.

Our differentiator is beyond the fact that Cohere does a lot of custom modeling and works very closely with Fortune 500 companies to get their AI needs met through modeling. North is also privately deployed in secure environments. So the very big companies, discerning Fortune 500 hundreds are highly enticed by that.

Which of Exa or Parallel (or both) do you use, and how long have you been using them?

While we use Tavily at Cohere, specifically for our web search needs, I'm on top of the industry. I'm familiar with these products, and I actually use Parallel through Manus, which I use heavily to conduct my own deep research. So Manus uses Parallel. I'm familiar with Parallel through not only evaluation and keeping on top of the industry, but also through using Manus more for my personal stuff.

What specific research problem led you to start using Parallel via Manus, and how did you handle that work previously?

As this industry has evolved, the needs have stayed roughly the same. Being able to conduct better or more thorough deep research has been a need since day one.

How Parallel plays into the stack with Manus is that Parallel is really the backbone of the deep research capabilities with Manus. And to be frank, they are fantastic. What Parallel offers, again, through Manus—I do want to clarify that I can't speak too strongly to the Manus stack and what they're doing that's adding a little bit of secret sauce—but I do know that they're using Parallel as their deep researcher.

Parallel is doing really well with long form comprehensive research—very multi-hop, very long detailed plans, and the results are fantastic. The grounding is very good, always detailed, responsive grounding. I really like how Manus uses that grounding to produce that in the interface. The results are truly best in class in terms of deep research.

Can you walk me through a specific research workflow where Parallel-backed results stood out, including the agent's steps and the final deliverable?

I do this all the time. For example, I recently helped a friend who is thinking about relocating out of the US to Europe. When I do cost of living analysis comparisons—what would it look like to live in cities like Barcelona, cities in France like Paris or Aix en Provence, or even Lisbon—the steps that the model takes is first and foremost, what I love about how Parallel works with Manus is that it educates itself.

It's not jumping into researching the cost of living in Barcelona. What it does is it first understands the parameters of what relocating looks like, what a person's needs are. And then once it has this basic context on what I'm even asking, then it starts actually researching the cost of living comparison. And it goes way deeper.

I'll oftentimes ask it to do a breakdown by segment. So not just a generic "it's 10 percent or 50 percent cheaper to live in Barcelona." It will go one by one. If you want school—private versus public, what does that look like? What are the offerings? Rent, in what area? What's a normal rent? What would be considered a luxury rent? Food based on lifestyle choices. It really goes deep. Usually, it uses a lot of tables, good pros and con tables, really good breakdowns.

How long does a deep, granular report like that take compared to your prior manual approach—minutes versus hours?

Using LLMs for deep research can be a double-edged sword. The research is obviously conducted much faster than I can do manually. However, then there's the part of validating the responses. If you don't do that, you might get yourself in hot water. You might say things that are incorrect or have beliefs that are not true.

While I do think that Parallel offers superior grounding and reliability to most other platforms that I've seen, it's not perfect. We're still talking about semantic relevance. We're talking about machines. They're never going to be 100 percent accurate.

In terms of time reduction compared to manual research, obviously, it's orders of magnitude shorter. Compared to other deep research platforms, interestingly enough, Manus and by extension Parallel, take much longer. It takes a long time to get an output. However, the level of depth is also unparalleled. So I would argue that while the time is significantly higher, it is a much better outcome.

Usually, to get an end-to-end research with a big meaty research question that requires a significant amount of research—like comparing cost of living with all these parameters and breaking it down—it takes somewhere between 10 and 15 minutes to complete.

When Parallel/Manus isn't perfect and you need to validate outputs, what do typical misses or errors look like?

It is quite good, so I need a moment to think about some examples. What comes to mind is that sometimes—and I actually don't even know if this is Parallel or really just the LLM itself (I believe they're using OpenAI tools, but I'm not sure about that)—the biggest miss is going to be leaning into leading questions.

Unless I'm prompting very carefully, with caveats and encouraging the model to push back or ask clarifying questions, it may just go along with my assumption. So if I have a leading question, like "explain why Barcelona is cheaper" rather than "do a cost of living analysis," it will go along with it, which then validates assumptions that may be incorrect.

What led you to evaluate Exa and Parallel, and how do they differ from each other and from Tavily?

In terms of evaluation, I haven't conducted a full scale evaluation on Exa or Parallel for Cohere's purposes to replace Tavily. We actually did do an evaluation and looked at a few providers. We used to use Brave, and then we landed on Tavily about a year and a half ago.

In some ways, these platforms are very similar. The differentiation they're offering is kind of case by case dependent. Meaning, I wouldn't see these platforms as wholesale different from each other based on certain few things. It's more like, if you have a specific use case, based on your use case, one platform might be a little better for you than the other.

What I think in terms of Parallel is they're taking the approach of building research and web infrastructure for AI agents. Whereas Tavily is a little bit more focused narrowly on just being a search engine results page API and additional grounding mechanisms, search mechanisms. I think Parallel is going for a slightly broader mission around deep research infrastructure and generally web infrastructure for AI agents.

I think Exa is similar to Tavily. Looking for how to power search. If I'm not mistaken, Exa's offering a huge vector database that passes a big corpus of most of the web. So they're going hard on the semantic similarity there. Again, that's going to be very use case specific. When we're implementing this as a lab who's building the entire orchestration from scratch, that's not necessarily a differentiator for us. But for other companies where this is not the key ingredient of what they're doing, it might be very helpful.

When replacing Brave, what friction points drove you away from it and what made Tavily the better choice over Exa?

So Tavily is spelled T-A-V-I-L-Y, just for clarity there. Why we looked at Brave versus Tavily is a great question. At the time, this is a year and a half, maybe 2 years ago, Brave wasn't yet built specifically for AI agents. It wasn't about LLMs. It was still just offering a search API with their own index. They were just basically competing with Google and offering an API behind it.

We needed to do a decent amount of transformation. The way results came out of that API didn't necessarily provide us all the text that we need. Keeping in mind that an LLM looks at text—it needs the actual text from the website, not a URL. So if we get a URL, we need to either do another hop to then go and retrieve the text from that URL, or if it's giving a description of a site or a synopsis, we had to do a decent amount of manipulation on the result and additional work to actually get the relevant text from the website that we believe is relevant and then use that to create the grounded response.

Tavily was doing all of this for us, and while quality was very similar, there was no significant change in quality or degradation. So it made it easier to integrate with Tavily. And frankly, we also had a good relationship with them, so that was another driver there.

How do Tavily's or Parallel's pricing and cost models map to the value you get—per-query versus subscription—and how does that scale with large volumes of users?

Specifically because Cohere offers the North platform as a privately deployed thing which means this lives in the customer environment, they need their own API key. So for Tavily, each one of our customers has their own Tavily API key. They are the ones who are paying directly. This isn't something that's paid through us. They're paying Tavily directly. And that also means that their volume is per themselves.

So even though we're talking about big companies—tens of thousands of employees, even hundreds of thousands of employees—at that rate, the cost is a more negligible thing.

However, if you are Manus or you're OpenAI, or any one of these SaaS big providers that have millions, hundreds of millions, or billions in the case of OpenAI, users, the cost becomes significant very fast. So if you are offering some kind of SaaS offering and you have reached some critical mass of users, even just something like hundreds of thousands of active per month, the cost becomes big.

I think, if I'm not mistaken, Parallel is also cheaper than Exa and the majority of players actually out there. I think they're offering a lot of good value. That's definitely important, specifically not for Cohere.

If Parallel and Exa are strong, what prevents OpenAI or Google from building the same web-infrastructure for agents and displacing these startups—how do you assess that platform risk?

I don't think they have a strong moat. That's a very good question. I think OpenAI building this feels like a distraction. If they're going to build something like this, then I would ask what else are they deprioritizing to get this done and why? Because that seems to me like just not necessarily the most important thing for a frontier lab to do.

Unless, for Perplexity, it makes sense—that was their whole value prop to begin with. We're going to be the AI search thing end-to-end. I could be convinced as to why an OpenAI would want to build that.

But you mentioned a much bigger, more important player in this space, which is Google. Why would Google not go and do this? I don't know why they're not. I do know there are parts of the stack that you can do with Google, like Vertex AI. But I don't think there's a moat. If Google decided tomorrow that they want to eat this industry up, they would do that without any problem. There's not any really meaningful secret sauce that is non-replicable that these companies are offering.

If there isn't a strong moat, why are Exa and Parallel gaining traction—are they simply faster on developer experience or doing something different with indexing or ranking?

This whole search for AI is a very brand new segment. We're talking about companies that have existed for less than 5 years. I think Parallel, if I'm not mistaken, was founded in 2023, so two and a half years ago, and Exa's in 2021. So we're talking about brand new baby companies. While they're getting great valuations, that's frankly just the AI bubble. I don't know their financials, so I'm not going to speak with confidence on that at all.

But I do know that right now, there's just this very strong mad dash to get any part of the AI pie. Why is Google not competing with them directly? I don't know. It might just be too small for them, or it might be something that is being cooked up. I wouldn't be shocked if Google does offer this in the next year.

There's also a rising tide bringing all ships up. Even as we talk about moat—that's a very good and important strategic lens when we're looking at these companies' valuations and long-term profitability or growth—but at the same time, we are still very early in the days of AI adoption in general, and definitely early in the days of search being used for deep research and agentic workflows. Every day, there's more adoption, more usage.

So whether the market share shifts negatively for one of these players, I still think they will be growing for the foreseeable future, at least for the near and mid-term future. Long term, do they have a chance to genuinely become a profitable company with a differentiated product and product strategy? TBD.

What one improvement would you most want Parallel or Manus to make—speed, domain-specific knowledge bases, or something else—to make deep research indispensable?

I think the answers to this question will have to do a lot more with Manus than with Parallel. Obviously, speed and latency—making it faster is always going to be good.

What would be nice would be if the way Manus leverages Parallel would include more domain specific segmentation around knowledge bases. What I mean by that is, when I'm asking for research, depending on what I'm asking for, it might need different web tools of domain expertise.

My fiancée has a medical condition. Oftentimes, I'm doing research using Manus to find answers to evaluate things. For example, I was looking at a specific supplement and wanting to understand which version of this supplement has shown better clinical results for her specific medical condition.

In this case, Manus using Parallel is doing a good job providing medical publications. But the way it's looking for them is through the open web. I can see what it's doing—it's using Parallel, searching, browsing, going to the website that is hosting the study, reading the study. But I would like it to have dedicated access to medical journals, not through searching the open web and then finding it. Having that more specific knowledge base would increase results.

And then using those different knowledge bases for different queries. If I gave the medical research question as an example, there are also different things like financial questions about taxes or laws. A lot of answers are out there on the web. However, the web is being crowded with AI-written content that is just a remix of something, and we're all just reading the same blog remixed 1000 times. The only reason the content's even out there is really for SEO for the publisher. So law offices will publish things about laws, which might be okay, but I'm looking for case studies, precedents, deeper research.

That's not to say that Manus isn't finding some of this. But I genuinely believe that it would be faster and better if it had dedicated knowledge bases that it could traverse for specific use cases.

If Parallel or Exa offered specialized connectors into proprietary knowledge bases, which domains would you prioritize for Cohere or your personal research—technical docs, financial filings, real-time API data, or others?

I think Exa definitely does this. I know that Exa has their domain specific streams. I'm just not sure about Parallel offering that. I think this could be valuable. Again, with Cohere being all on-prem, very customized for customers, the way I would prioritize this is on a customer by customer basis.

There's definitely room for this. I think it would be absolutely valuable to some of our customers to integrate into something like this. I know Exa has a financial stream specifically for financial filings, analyst reports, things like that. A lot of our customers are financial institutions, I think that'd be great value.

Has customer interest in Exa-style domain streams come up at Cohere as a differentiator from generic APIs like Tavily?

Yes and no. We have discussed things like this at Cohere. It's something that is on our roadmap, but I can't speak too much about Cohere's roadmap. The thing is that our customers' priorities lie elsewhere.

If you asked if customers are asking for this—no. But I'm not sure that's because they don't need it or want it, but rather because they don't even know it exists. We're still living with a big information knowledge gap in AI. Even when I talk to other people in the industry, our customers are often the machine learning folks for AI. There's so much to learn and everything's changing so fast. So I can't say that because customers are not asking for it, that's necessarily indicative of a lack of need.

In terms of why this is not a top priority for Cohere today—most of what we're doing right now is focused on getting the internal document grounding strong, and that's really the primary use case. While web search and these domain specific data streams could have a lot of value, the real big differentiator and what Cohere is really trying to accomplish with its customers is getting that internal knowledge to be very dialed in.

And that's very hard to do. Connecting into these proprietary sources—there's a lot of infrastructure that goes there, a lot of sophistication that needs to be built, a lot of evaluation and a lot of skewing customer by customer case. So just getting that really good—that's our primary focus rather than extending the web search or introducing new data sources. These Fortune 500s have mountains of information that they need to traverse to get these answers. So that's really what we're focused on.

If Exa or Parallel built superior connectors for internal enterprise data that competed with Cohere's RAG infrastructure, would you view that as a threat or a useful utility?

Both, it depends. Cohere acquired a company that does these connectors. So it's definitely something that can compete to some degree, and it's a utility.

Specifically taking the lens of Cohere, it's not a threat really, because a lot of the data connectors that we need are for proprietary systems or proprietary implementations of well-known systems. For example, SharePoint—Microsoft SharePoint is this huge complicated thing that every customer that has it configures very differently. The data they store there, the formats they store it in, and what's in these documents—it's very different.

What Cohere is really offering is the hands-on implementation. That's not a SaaS tool—that's not something you can make into a SaaS tool. So for us, it's not a very strong threat. It might be a utility if they do something in the stack better and cheaper than us. That's fantastic.

As an investor advisor, what key risks—beyond Google or OpenAI entering—should be flagged for Exa, Parallel, or Tavily, such as technical or structural vulnerabilities?

I think the need for RAG and using external information will persist for the foreseeable future. There's a world where an LLM unassisted by external infrastructure, with minimal search results, would be able to produce relevant results. But that's not how it works. Like, no matter how smart the brain is, it's always going to need external information unless it's connected directly to a database of all human knowledge, which is not realistic.

So I don't see the problem coming from model improvement—quite the opposite. That's just going to make the use of these infrastructure tools better, faster, cheaper.

But what I will say is that I can't get away from the fact that Google or OpenAI moving in is the huge elephant in the room. I don't know what tools Exa and Parallel are planning on using or how they're really planning on differentiating in a meaningful way.

If I were advising an investor, I would want to have a really strong understanding from these companies' CTOs or Chief Product Officers: How are they planning to differentiate? How are they differentiated today? And is that meaningful enough from a product strategy and a business strategy to not get swallowed up by Google tomorrow or OpenAI?

There are a lot of platforms already popping up more and more, which shows you that the barrier to entry is low. Exa is a Y Combinator company—they started 4 years ago, now they're raising a bunch of money. Great. But Tavily is an even smaller company as far as I'm aware, and they're doing pretty well.

Again, the rising tide is giving the impression that everything's incredible. But when there's more of a plateau on usage—which won't be tomorrow, not in a year, probably not even in 3, but when we reach a point of market saturation where everybody has their AI agents in any device they want—there's a question of whether I would choose one or the other. And at that point, these companies will really need to have strong answers.

What killer app at Cohere would gain orders of magnitude from a 15-minute Parallel-backed deep report versus a standard RAG answer—what concrete use case benefits most?

There's an assumption baked into your question, which is that North doesn't do that—and that's not the case. North definitely does significant multi-hop deep research and reasoning.

But what I would say is that a big use case for us is the financial analyst trying to understand companies and create a report. Having very strong reasoning to understand which tools will have which information and use them intelligently is very key, and that's something that we do have. The proprietary web search stuff really dialed in is just going to enhance and enrich that.

Where there's room for growth is not just helping you find the information you asked for and not just helping you use the different data sets correctly, but also offering improvements. This is more of a modeling thing, but I think the models would do better if they had higher quality results.

For example: "I made this report for you, here's the output. But also here's a list of suggestions for sections that you didn't ask for that I think you should include." Or "here are some interesting results from the web that I think would enhance the fidelity of your report."

A lot of this is more about Cohere's models, but I think if it had the relevant results from the web, it would be easier for it to make suggestions. If it knows what it doesn't know, then it can suggest additional resources to look at.

If Parallel or Exa charged much more for proactive, high-fidelity domain results, how would you evaluate that ROI and is there a ceiling you'd pay for better-than-Google results professionally?

There's always a ceiling. It's hard to answer that question considering how much of this is really more dependent on the modeling layer above it.

But let's say for the sake of this discussion if Parallel or Exa or anyone offered superior results—not necessarily superior in terms of the accuracy of search results, but something like "Here are the first 100 results that are most relevant to your query. It seems like you're researching XYZ based on the user prompt that the LLM produced. Here are additional supplemental pieces of information that the user should consider."

Let's ground this in something more real. An analyst is trying to create a report on Nike. Nike's reports just came out—here's the 10-K, the 8-K, all these filings, and we have all this information. I want to understand and create a market projection on Nike.

If the model was clever enough, and if it had access to this information easily (without triggering an additional search necessarily because Parallel was able to provide this layer), it might say "Here's a great report with all the financial information about Nike that you asked for. But here are high level summaries or pieces of information that would be useful for your report: context around regulations on cotton manufacturing in Pakistan that you did not ask for but is relevant to the forecast you're trying to make on Nike, or here is some relevant news today about adidas that may have an impact on Nike's stock."

That could be very useful. If the company was able to provide that as a standalone infrastructure layer, I'd be interested.

Would you value Parallel as a standalone analyst-facing product, or is its value tied to being embedded in an agentic orchestration layer like Manus?

The question doesn't make a lot of sense because whether the interface and the orchestration, reasoning, and planning is handled by Manus or handled by anyone else, I'm only concerned about that in terms of the quality.

If Parallel offered a competing product that was as good or better than Manus, then I would consider that. But I don't think they can do that—they can't offer the same value without building the same orchestration layer, the integration of the LLM, and additional tools. So if they want to compete with Manus, they would have to build all this stuff. It's not like they have a way to cut out Manus without recreating it.

When running lengthy Manus/Parallel reports, have you encountered latency, timeouts, or looping failures, and how are those edge cases handled?

This is more of a thing on the platform level, not so much the APIs. You get a call, you get a response, even if it takes time. Parallel has a research agent product, and I'm sure they have ways to handle errors and crashes and things like that.

On the user interface level—for us with North or for Manus—there are instances where things fail, whether it's a model sometimes spinning itself out a little. It goes a little too heavy on reasoning stuff and you get death loops. It's not often—it's the minority of cases, not the big majority. But I don't know that there's a one-size-fits-all answer to this question. Every engineering team is doing their own thing there.

Are there other startups in the AI research space you've watched that do something fundamentally different from Exa, Parallel, or Tavily?

I don't have a really strong answer for that. I do think Bright Data has an Unblocker API which is kind of designed for bypass, and I think that's pretty cool. Especially as the whole internet has always been designed around "you're not a bot." And now it's like, well, you are a bot. So Unblocker API is pretty neat. I don't know if Parallel offers something like that. But otherwise, I think that's something cool that Bright Data is offering.

How do you think about the economics between inference (runtime queries) and training/data (scraping/pre-canned datasets) when evaluating higher-priced domain results?

There are really two sides of the AI core that interact with these platforms. There's the inference side—at runtime answering a user query. But then there's the training side.

These companies evolved into offering as the big consumers of data, these pre-canned datasets. They also have just massive scraping needs, and that's a very different use case, but obviously the same companies. Cohere needs data curation, scraping, whatever it needs. But it also has tools for users.

So I'm living on the platform side, but there's an entire side that does the model building and training. That's just something worth noting. When you're talking to investors, think about that—there are really two sides to this, and different companies might offer different tools for the two different sides.

I haven't thought much about which side is more lucrative, and maybe for the consumer it's more important when you think about these companies. But it's definitely something to think about in the landscape. I would also look into end categories for this, like how much is OpenAI and others paying for scraping and data companies versus how much are they paying for user-generated queries in real time.

What would need to change for Cohere to bring web crawling and search fully in-house instead of using vendors like Tavily—maintenance burden, performance thresholds, or something else?

Nothing is easy. Each of these companies has done meaningful work that wouldn't be trivial to replicate. It has nothing to do with that. We are actually the best in terms of embeddings—our search abilities are fantastic.

If we really wanted to enter the industry in terms of search, that's completely sane. Would I ever do that? I have so much other stuff that's going to deliver real value to my customers. Replacing a vendor is not top of mind.

Maybe for OpenAI, I could see a world where they have a reason to do that. The math probably makes sense for a consumer-facing free platform. The economics get in sync. Get a team of 15, 20, 30 engineers, and all that still makes sense because at the end of the day, they'll save millions of dollars. Okay, sure.

But for me, it's not worth it. And as I said at the beginning of the call, I would much prefer to pay a vendor to do a good job rather than for me to do a job that is not part of my core value proposition.

This transcript is for information purposes only and does not constitute advice of any type or trade recommendation and should not form the basis of any investment decision. Sacra accepts no liability for the transcript or for any errors, omissions or inaccuracies in respect of it. The views of the experts expressed in the transcript are those of the experts and they are not endorsed by, nor do they represent the opinion of Sacra. Sacra reserves all copyright, intellectual property rights in the transcript. Any modification, copying, displaying, distributing, transmitting, publishing, licensing, creating derivative works from, or selling any transcript is strictly prohibited.