Improving search with RAG architecture with Pinecone CEO Edo Liberty

**Sarah** (0:05)
Hi, listeners, and welcome to another episode of No Priors. Today, Elad and I are talking with Edo Liberty, the founder and CEO of Pinecone, a vector database company designed to power AI applications by providing long-term memory.
Before Pinecone, Edo was the Director of Research at AWS AI Labs and also previously at Yahoo. We're excited to talk about the increasingly popular RAG architecture and how to make LLMs more reliable. Welcome, Edo.
Let's start with some basic background. Can you tell us more about Pinecone for listeners who haven't heard of it? What does it do and how does it differ from other databases?

**Edo Liberty** (0:40)
Pinecone is a vector database. What vector databases do very differently is that they deal with data that has been analyzed and vectorized, I'll explain in a second what that means, by Machine Learning models, by large-language models, by foundational models and so on. The most large-language models or foundation models, actually any models, really understand data in a numeric way. Models are mathematical objects. When they read a document or a paragraph or an image, they don't save the pixels or the words, they save a numeric representation called an embedding or a vector.
That is the object that is manipulated, stored, retrieved, and searched over and operated on by vector databases very efficiently at large scale.
That is Pinecone. When we started that category, people called me concerned and said, what is the vector and why are you starting a database? Now, I think they know the answer.

**Elad** (1:49)
How did you think about this early on? Because you started the company in 2019
At the time, there is wave of generative AI hadn't happened quite yet. I was wondering what applications you had in mind, given that there's so much excitement around Pinecone for the AI world, the prior AI world had a slightly different approach to a variety of these things. I'm just curious, were you thinking of different types of embeddings back then? Were you thinking of other use cases? What was the original thinking in terms of starting Pinecone?

**Edo Liberty** (2:17)
The tsunami wave of AI that we're going through right now didn't hit yet, but in 2019, the earthquake had already happened. Deep learning models and so on have already been grappled with. Large language models and transformer models like BERT and others started being used by the more mainstream engineering cohorts.
You could already connect the dots and see where this is going. In fact, before starting Pinecone, I myself had found our anxiety between are we already too late, versus nobody knows what the hell this is and we're way too early. It took me several months of wild swings between those two things until I figured, maybe the fact that I have those too early, too late mood swings maybe means it's exactly the right time.

**Sarah** (3:07)
Maybe, Edo, you can actually just expand a little bit about in what use cases people want to use embeddings.
I think there are ways to interact directly with language models and then reasons for, for example, reliability or context length and performance of people interact with embeddings in a rag architecture or in semantic search. Maybe you can talk about some of the driving use cases.

**Edo Liberty** (3:35)
I mean, the obvious way in some sense to add knowledge to your conversational agent, whether it's chat or what have you. We talk about it as generative AI now, but it's much more general than that. Is to, again, not shockingly, bring the relevant information into the context, so that you can actually arm the foundational model with the right pieces of content, with text, with images, with what have you.
You want to be able to retrieve that from a very large corpus of knowledge that you have, whether it's your own company's data, or whether it's the Internet, or what have you.
It so happens that LLMs are already very, very good at representing data in the way that they want to consume it, which is these embeddings. You can at question time, in real time, at the time of the interaction, go and find relevant information. Irrelevant might be associated with, or collated with, or something that is similar to whatever it is that you're being asked about. Once you bring that into the context, you can now give much more accurate answers. As a side experiment, we actually loaded, it's called Common Crawl, which is the top Internet pages crawled fairly frequently. We load that into Pinecone and saw what happens when you augment GPT 3.5 and 4 and Lama and Mixtral and models from Coherentropic.
You could see that if you augment all of them with RAG, even on the Internet, which is data that they were trained on, you can reduce hallucinations significantly up to 50 percent sometimes. Interestingly enough, many of them actually start behaving quite similarly in terms of level of accuracy, even though without RAG, they actually have quite different behaviors. So it's both like a uniform improvement and a little bit of leveling the playing field. Now, because we know we can do that very well now, now you can do that also with proprietary data, with data inside your company and so on, stuff that of course is not available on the Internet and stuff that those models were never trained on. And interestingly enough, again, the quality ends up being incredibly high.