The AI Doctor Can See You Now, with Vivek Natarajan and Khaled Saab from Google

**Erik Torenberg** (0:00)
Hey everyone, Erik here.
At Turpentine, we're building the first media outlet for tech people by tech people. We produce the show you're listening to right now. We're also building a network product that is bringing together the best founders and executives in the game, from all stages.
The Turpentine Network is a high trust space where exceptional people can talk. We already have over 400 members and growing. The community benefits include tactical advice, tech stack recs, hiring referrals, in-person events, an investor database, and exclusive perks. If you want to join, you can apply at the link in the description.

**Nathan Labenz** (0:35)
Hello, and welcome to the Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello, and welcome back to The Cognitive Revolution. Today, I am thrilled to share my conversation with Khaled Saab of Google DeepMind and Vivek Natarajan of Google Research. This is Vivek's third appearance on the show, the most of any researcher, and for good reason. As regular listeners will know, when I'm asked why we should be excited about AI, my standard response is to point to the incredible potential value of AI doctors. In a world where access to medical professionals is all too scarce, even in rich countries, and so much more in poorer parts of the world, the prospect that anyone globally, one day soon, could have access to high-quality medical advice from their personal device anytime, day or night, for less than 1% the cost of current first-world access, that is all simply too valuable to ignore. And the good news is that Vivek, Khaled, and their colleagues at Google who focus on the medical applications of the latest AI models have made extremely impressive progress over the last 18 months, demonstrating that with a mix of techniques, including strategic data curation and filtering, repeated fine-tuning, uncertainty modeling, and painstaking evaluation, large language models can be effectively adapted to medical applications, including radiology, diagnosis, multimodal understanding of medical records, and many more, often rivaling and increasingly even surpassing human doctor's performance on the exact same tasks.
That is exciting stuff. But what's even more exciting about their work today, to me, against the backdrop of continued hyper-scaling and the prospect of an international AI arms race, is how their results demonstrate the transformative value that we can already achieve with current models if people are willing and able to put in the hard work needed to dial in and validate performance. Of course, it's undeniable that each new generation of model brings greater capabilities and makes application development easier. But it's worth noting that some of the human competitive results we discussed today are based on the Flamingo model, which Google originally published more than two years ago now in April 2022 And I was fascinated to hear Khaled and Vivek tentatively forecast that they could probably achieve their vision of a high quality general practice AI doctor, even if Gemini 1.5 Pro were the most powerful model that they ever had the chance to build on. This to me strongly suggests not only that current models are indeed in some sort of a sweet spot, where they're powerful enough to be extremely useful, but not so powerful as to risk catastrophic harm, but also that we can afford to move with caution through further orders of magnitude of scaling, confident that we won't be leaving all the value we hope for on the table. In other words, there really might be solid ground from which to defend my adoption accelerationist, hyper-scaling pauser position. While Khaled and Vivek are proceeding very responsibly and methodically, cognizant of the fact that people generally need overwhelming evidence before they'll be comfortable trusting AI systems in critical contexts, I personally would advocate for a warp speed project for AI doctors, powered by 10 to the 26 class models, while the new sciences of interpretability and AI control are given time to develop.
In my wildest dreams, that might even be a joint project that we could work together on with China. I'm really grateful to Khaled and Vivek for joining me and for the amazing work that they are doing. I truly believe this technology will save many lives in the years ahead. And I hope that by spotlighting it, I can help inspire others to work on high value applications of today's technology, rather than waiting for further scaling to solve all of our problems.
As always, but even more so for this conversation, which I think is extremely important, I would appreciate it if you'd take a moment to share the show with friends. Know too that your feedback and suggestions are always welcome either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network.