Long-Term Memory for LLMs, with HippoRAG author Bernal Jiménez Gutierrez

**Erik Torenberg** (0:00)
Hi, everyone. Excited to announce a new podcast that just launched from Turpentine. Complex Systems with Patrick McKenzie. Patrick, who is better known as Patio11 on the internet, thinks a lot about systems, software, financial infrastructure, and so on.
If you're tired of hearing that everything is broken, this podcast is for you.
Patrick surfaces conversations with experts who actually built and understand the complicated but not unknowable systems we rely on. You might be surprised at how quickly Patrick and his guests can put you in the top 1% of understanding for stock trading, tech hiring, and more. Subscribe to Complex Systems with Patrick McKenzie everywhere you get your podcasts or at the link in the description.

**Nathan Labenz** (0:41)
Hello, and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello, and welcome back to The Cognitive Revolution. Today, my guest is Bernal Jimenez Gutierrez, PhD candidate at Ohio State University and the lead author of HippoRAG, a novel approach to retrieval-augmented generation inspired by the human hippocampus.
RAG, of course, has become one of the most important techniques for grounding large language models in specific knowledge bases. But as anyone who's implemented a RAG system knows, there are still major challenges, particularly when it comes to questions whose answers are not recorded explicitly in a single document, but have to be inferred from a more diffuse collection of sources. That's where HippoRAG comes in. Drawing inspiration from the hippocampal indexing theory of human memory, Bernal and his collaborators have developed a system that uses LLM-powered entity recognition and embedding clustering synonym identification techniques to pre-process knowledge into a graph structure, which can then handle complex queries requiring multi-hop reasoning far more quickly and affordably than previous state-of-the-art approaches. In our conversation today, Bernal walks us through the neuroanatomical inspiration for HippoRAG, explains how it works under the hood, and discusses its current capabilities and limitations. We also brainstorm ways that it could be extended and improved, considering how it might complement other recent advances like Raptor, as well as the strategy of periodically creating synthetic reflective memories, which I first encountered in the AI Town project.
Notably, it seems to me that these techniques are pretty much conceptually orthogonal, such that an application developer might be well-served to implement all three, and perhaps even more, into a single RAG system. Assuming, of course, that the use case justifies the use case.
As always, if you're finding value in the show, we'd appreciate it if you'd take a moment to share it with a friend. We welcome your feedback via our website, cognitiverevolution.ai, and you're always welcome to DM me on your favorite social network. Finally, for now, a programming note. I'll be away for a short vacation and to speak at the Adapta Summit in Sao Paulo for the next couple of weeks. But don't worry, we've got a full schedule queued up for while I'm gone, and I plan to make a big push to generate more connections between AI engineers and advisors and actively hiring companies when I'm back. So definitely remember to submit your resume if you're interested in being a part of that. Now, I hope you enjoy this fascinating exploration of biologically inspired AI memory systems with Bernal Jimenez Gutierrez, creator of HippoRAG.
Bernal Jimenez Gutierrez, author of HippoRAG, welcome to The Cognitive Revolution.

**Bernal Jimenez Gutierrez** (3:39)
Matt, thanks so much for having me. I'm excited.

**Nathan Labenz** (3:42)
I guess for starters, two things caught my attention about this project. One is just a performance, which we'll get into and the fact that it works and seems very practically useful. The other is that you're drawing inspiration from biological systems to kind of motivate the design of this new RAG system. So as the name would suggest, the project is inspired by the Hippo campus, which I think is cool.
You want to just kind of give us a little bit of background of, first of all, who you are, the group you're working in, and then specifically for this project, like what was the kind of original motivation? What was the bug that got you to dive deep into this?

**Bernal Jimenez Gutierrez** (4:18)
Yeah, sure thing. So I'm entering my last year here at Ohio State. I'm working with Professor Yusu at the OSU NLP lab and our group does a lot of different things, from embodied agents to more like mind to web came out of our group, so like agents in the web. And Dr. Yusu is really interested in the biological inspiration side, so that also is a big part of our group. But me in particular, I graduated from Berkeley in 2015 with an applied math, BA, and I started working at a healthcare startup that did NLP for matching cancer patients with clinical trials. And I really got into this biomedical NLP direction of research, and I've been doing that for many years, using AI to facilitate biomedical research through NLP tools. Well, in the past, it was just like ontology's knowledge bases, like UMLS, and I've been kind of transitioning into how do we really use LLMs effectively in this biomedical NLP space.