When AI Improves Itself | Richard Socher (Recursive)

Richard Socher (0:00)

Anything you can simulate, AI will solve. But before you know it, you're in this Recursive Self-Improvement Loop, and we believe that that will be a great unlock. Boy, are we far away from the true upper bounds of any of the spaces of intelligence, and there's still so much further that AI can go.

Matt Turck (0:19)

Hi, I'm Matt Turck, welcome back to The MAD Podcast. My guest today is Richard Socher, one of the most cited researchers in AI, and now at the center of the term, everyone in AI is suddenly talking about recursive self-improvement, AI that makes better AI. Richard just raised $650 million for Recursive, a company built to do exactly that, and his new book, The Eureka Machine, is a fascinating blueprint for how AI and RSI are about to revolutionize science. Please enjoy my conversation with the always excellent Richard Socher.

Hey Richard, welcome back.

Richard Socher (0:54)

Great to be back, thanks for having me.

Matt Turck (0:55)

All right, so let's to catch up on, we're going to talk about Recursive Intelligence, we're going to talk about Recursive the company, but first and foremost and most importantly perhaps, we're going to talk about your new book entitled The Eureka Machine, which I read with great interest and would strongly recommend coming out in a couple of weeks, I believe. The book opens with a premise that I think a lot of people find surprising and shocking, which is this claim that scientific progress has slowed down, which feels counterintuitive given the number of researchers we have around the world and the sheer amount of money that goes into the space. So why is that?

Richard Socher (1:39)

Yeah, it's a somewhat surprising fact and you may argue, clearly not. We're making so much little progress on so many different things, but when you think about how much progress have we made on antibiotics, bacterial infections went from a death sentence and the plague to a nuisance. We now have antibiotics for almost all the different bacteria, and we truly solved that.

We have clearly not solved viruses or cancer the same way we solved bacterial infections. When you think about foundational novel things like E equals MC squared and general relativity, we clearly have not made progress on many theories and physics either when it comes to such foundational things that then literally led to nuclear energy and fusion and fission and other kinds of research that could be conceptually done. And so a lot of the fields have kind of gone through, from we understood some foundational pieces, to we can now do a lot of engineering, but they've also split up into thousands of different subfields. It is almost impossible nowadays to be the sort of general genius that can dabble in all of these different fields, because each field takes years and years and years to get really deep into. And so what we found is that as there are more and more subfields and niches, it's actually hard to have enough people in each of these subfields. And Stanislav Lem and others have talked about that and predicted that that will be a big part of why we're slowing down.

Matt Turck (3:08)

Yeah, you have a great expression. You talk about how we evolved from a body of knowledge to a labyrinth of knowledge, 34,000 journals that might as well have no trespassing signs.

Richard Socher (3:20)

That's right. Yeah, it's so hard to even understand all the lingo.

And I've sort of gone through this myself first when I started studying linguistics and computer science. But now that I'm sort of trying to study and have studied now over the last few years sort of on the side biology, it's like, man, every time you have a conversation with biologists, like 10 sentences in, they're just telling you so many abbreviations and terms that you're not familiar with, that most people after a while just space out. And so it's really hard to describe and explain very complex deep fields to someone who hasn't been in them.

Matt Turck (3:56)

You also mentioned that there is some level like social human element to this, where in academia you're not necessarily encouraged to take risks. And that goes to your own experience as well?

Richard Socher (4:08)

Yeah, 100%.

People often, the way careers work is you want to be kind of novel, but if you're too novel, if you're too far out there, then your papers will get rejected. And that certainly happened to me a lot in the early days, like 2010, of neural networks for natural language processing, where the majority of my first couple of papers got rejected from NLP conferences. When they got accepted in some small sub-niches and subgroups, and I still remember the first deep learning workshop at NIPS back in the day, now NeurIPS, that was basically 30, 40 people, all the now super famous folks, but it was just a couple of us renegades who thought that this would clearly be the right way of going about it. And we came from different directions, like feature engineering seemed not the right path to doing things. Feature learning was a big part of what got us started. Some were neuroscience inspired, and it was again that sort of combination of different fields that was at that intersection where it's interesting, but also often hard to publish well in the beginning.