Jeff Tang, CEO of Athens Research, on Pinecone and the AI stack

The rise of AI that we've started seeing—especially last fall and winter when ChatGPT was released—was part of the catalyst for me stopping working on Athens. It seemed like these unstructured approaches to data capture and retrieval and all the other things, like summarization and all those facilities, just were going to leapfrog a lot of the ways people were using note-taking apps, which just felt so tedious and manual in comparison.

Permalink

Could you say a little more about your previous work with Athens? What did building on a graph database at that point allow you to do with that company?

From the developer/builder’s point of view, graph databases, like a lot of NoSQL databases, are just very flexible. They allow for a faster iteration cycle. The end product we were providing was more of a graph being surfaced to the users. That’s what was interesting or unique about Athens. We were providing features like bidirectional links, properties, relationships, and nodes to users as an end user feature, which hasn't really been done up until this point for your everyday consumer.

A bidirectional link is similar to a relationship that you might see between graph theory. It's just a nice way of organizing stuff for people that are generalists—let's say like a product manager who has to work with designers/developers as well as a go-to-market team. A system like Jira is not really conducive to that—it's more for bugs and tickets and low level issues. Then design doesn't even really have their own project work system, so things tend to default to Google Docs or Notion or Slack and really long Zoom meetings or something like that. 

But when you have something that's touching on a lot of different fields or disciplines, having a graph is a really helpful way of organizing pretty much heterogeneous data types and data sources.

Permalink

So your conviction about AI hinges on AI including vector databases being potentially a better way to accomplish the goals of that company note-taking, note surfacing etc?

It's not even at the level of note-taking. I would say that's something that people get wrong about Athens. Maybe for some of our competitors that was true, but the point of Athens and other tools for thought, you might say, is not to just be better at note-taking. Note-taking is this sub-secondary thing or something that's upstream of solving some other job. Or it's just a workflow in a process of thinking and trying to problem solve. 

When you see a note-taking app, it's really more about improving cognition.

AI is extremely capable of doing summarization or finding a common thread—not so much generation, like coming up with 10 different AI startup ideas, but more so let me work with this text and slice and dice it as if I had a pretty smart idea-sparring partner or a really good rubber duck.

It felt like, again, in an ideal world, maybe no one takes any notes at all. That's what they mean by leapfrog, which is, oh, the AI isn't going to help us to do better notes. It's just that we don't even need to take notes. The point is Athens was trying to solve it. Note-taking sucks a lot of the time and can we make that better? But if you don't need to do it at all, that's great. Then you get into fuzzy territory of, ‘oh, yeah, we'll just let the AI do everything for us.’ Maybe that's too big a leap, so far, at this point. But, yeah, it's all about helping, for now until we have AGI that solves all our problems for us, let's just use it as a tool for augmenting humans and our cognition.

Can you talk a bit about the stack you built for medicaltourism.ai with—Pinecone, LangChain, Vercel, etc? Can you describe what your stack looks like and what are the most important tools in it?

My stack is LangChain, Pinecone, Vercel, and Next.js for web app development—and that’s basically it. I didn't have much machine learning AI experience before this. I'm actually a coding bootcamp graduate. I don't even have a CS degree or a degree. That's probably the whole cool thing about this, which is that AI is being democratized.

Permalink

Is LangChain kind of this Rails-esque framework that lets you get started more easily?

Obviously Pinecone and LangChain make it easier to get started with AI. But I don't think Rails is a good analogy because Rails is very opinionated and locks you into their way of doing data modeling and application handling. Whereas, what's good about LangChain is that it's actually abstracting away the LLM or the vector store. You're not tied into OpenAI's API. You're not tied into Pinecone. You could use FAISS or an open source alternative. Or you could use Anthropic or the other LLMs. 

In that way I see it more as a middle layer, sort of like Terraform if you want. With Terraform, you're not tied into AWS or Azure as a tool—you use them interchangeably. Or that's the dream.

That's why LangChain is cool—you're not tying into OpenAI or Pinecone, in particular.

How do you think about interoperability and being able to switch between different model providers? Is it similar to the way a SaaS app today might look at changing between different cloud providers?

It really feels like OpenAI is just way better than the rest for the time being. I haven't even looked at the other LLM APIs. That being said, as a developer, I never want to feel like I'm locked into something. That's why a lot of people like open source alternatives, like Athens or others. In practice, I don't want to self-host this because it's so much pain or work. But I know I can have an ejector seat in case I need to get out. The AI landscape is really crazy right now. Maybe someone will overtake OpenAI in a few months and then all I have to do is change the import statement and then, ideally, that's all I have to do.

Why use Pinecone over other vector databases?

To be honest, I don't think that much when I'm choosing my tech stack. I just use whatever is popular, or what people talk about the most, because I'm too lazy to think about the technologies from first principles. Also, because I'm not building something for scalability. I'm building stuff for prototyping. If everyone's talking about that, I'm just going to outsource my decision-making to what people are talking about. I've seen the LangChain guy multiple meetups and I talked to him and it's like, cool, it seems like this is trending. I'm going to make random experiments with AI to get some clout on Twitter using the popular libraries right now. That was basically it. 

In terms of why Pinecone over something that's free open source, it's like if I'm building something prototyping, it's almost always easier to use the hosted version. Because I'm not going to go over their quota and it's probably just going to work the best with the least amount of work. Then I can just, again, mention them in my tweet, launching it. Then they'll retweet me. That's what they all did. Then they DM me like, "Hey, we want to learn more about what you're building." It's like, okay, cool. Even this conversation, right?

You want to talk to me because I'm using Pinecone and LangChain, but I'm just using them because other people are using them. But I think that's so important, though, at the same time, which is that languages and technologies all have network effects, and it's not always the best product wins. Like Athens, our tech stack was, closure, closure script, data script, these functional programming languages, these graph databases, super esoteric stuff. I'm not using that anymore. Maybe there might be a use case in the future where this definitely makes sense to use. But, at the end of day, it's just easier to use TypeScript and Postgres and React because everyone's using that. It's like a self-fulfilling prophecy in a way because that momentum just makes everyone evolve that tool chain more.

Permalink

What are some of the major points of friction in the AI development workflow today where you see opportunities to make it better?

I have run into some issues using LangChain—just random errors that I feel like aren't my fault that I haven't looked into it enough. But I feel it's just normal issues, normal API issues, normal documentation. They're normal issues you get if you're using cutting edge technology. I have to read the source code sometimes because the code is more up-to-date than the docs but that's just normal for a startup. I still don't know the best way to deploy LangChain applications.

They have a list of ten different things and I used Vercel and it sucked. Every startup is trying to become the startup that people deploy LangChain apps on, so they're making a bunch of noise. It's not clear how to deploy, or the best way to deploy LangChain apps. But I didn't have too much friction. I ran into some errors when I was trying to build an agent this past weekend, where it's querying different APIs. But I don't really see that as a big issue. 

In terms of where could I see more startups being built and products being built for this ecosystem? I'd say, just looking at all the modules that LangChain has for agents or documents or vector stores. Pinecone has the vector store, but they have other modules, and I feel like all of those modules could probably build a few startups around there, or there will probably be a few startups if there aren't already. Just extending the capabilities and making it as easy as LangChain, basically. It just works. It's dumb, simple, stupid. Its hosting is free for a low quota.

Permalink

What is your take on Vercel's attempt to get into the AI deployment space?

Well, I just used the template in the LangChain docs. Then I added one dependency and then it was suddenly too big for the Vercel deploy to accept it.

That's just serverless functions, in general—so other products would have that same problem.

It's definitely an arms race right now to build the best AI tools. I don't really think of Vercel as an AI company. I just think of them as a deployment company. They're really, really good at it. They're like a web app company. But it'll be interesting to see if the AI native dev tools companies will win versus the incumbents.

I'd be interested to hear your perspective on using a bunch of these AI native tools like Pinecone and LangChain. What's the difference between a tool that's designed from the ground up with AI in mind, versus one like Vercel or Algolia, where they bolted on within the last three months to try and ride this wave?

It's hard to tell. In some ways, I don't feel like using LangChain or Pinecone is any different from using any other sort of SaaS tool or dev tool or API. Whether it's 10 bucks per month, or it's X requests for Y cents. That's why it's so accessible and awesome for me as someone that's not an ML researcher or engineer. But, at the same time, it feels like LangChain and Pinecone are just going to be more of a fit for me because they're focused on that entirely. That's all their branding and positioning.

Long-term, does that mean they'll be better? I don't know. It's kind of like they're spending all the money and stuff on events and marketing, so right now I feel like they're going to be better. I don't know long-term if they will be, though. I feel like there's an analog in the end users tools, as well. Everyone has AI features like Notion or Mem and all the other 19 apps that everyone's doing with plugins. They're all using the same features. They're all using GPT-3. 

At the end of the day, I don't use any of them for their AI features and I don't really know anyone who does. It is just more of a marketing thing I feel like. I don't know if we've really seen the rise of AI native dev tools, or AI native consumer tools because it mostly feels like marketing on both ends. There's more usefulness in the dev tools area for sure than the consumer tools.

Permalink

How do you differentiate trying to build a product at this phase in the generative AI market?

I think the question of differentiation is interesting, how to differentiate as someone trying to build a product in a market. All the AI founders, generative AI founders, probably get the same question from VCs like ‘hey, what's your moat?’ At the beginning, I definitely felt like this is a good question or I understood the question. But after thinking about it, I realized that there really aren't moats in SaaS a lot of the time. As someone who was building a note-taking app and there are a bunch of other note-taking apps. In productivity, there's a bunch of project management apps. Then it's like, ‘oh, wait, there's a lot of analytics apps, as well.’

There's a bunch of observability and monitoring startups. I don't know if any of them has a moat, for real, or it's just a pretty big market and it's not super winner-take-all. Why are there so many billion dollar project management companies? I don't know. What moats does SaaS have, in general? It's not really apparent. At the end of the day, execution is still everything. If you’re building the tenth customer support bot trained on docs, or the tenth startup doing writing or copywriting, which under AI, you just have to sprint, I don't know, and hope that you have the best founder-market fit, that you have the best distribution and data and brand. Those are the things that don't change as moats. 

So even for a lot of AI companies, I feel every time a new GPT gets released, a bunch of startups just die. That could happen again to all the ones being built on GPT-3.5 or 4, those could all go away by GPT-5 because the base model is just so much better. Or, actually fine-tuning and being an amazing prompt engineer doesn't really matter that much when it's just the bar constantly getting raised.

Permalink

This transcript is for information purposes only and does not constitute advice of any type or trade recommendation and should not form the basis of any investment decision. Sacra accepts no liability for the transcript or for any errors, omissions or inaccuracies in respect of it. The views of the experts expressed in the transcript are those of the experts and they are not endorsed by, nor do they represent the opinion of Sacra. Sacra reserves all copyright, intellectual property rights in the transcript. Any modification, copying, displaying, distributing, transmitting, publishing, licensing, creating derivative works from, or selling any transcript is strictly prohibited.