How Replication Could Teach Machines What Good Science Looks Like — Edward Hughes

Edward Hughes (0:00)

How do we start to build AI scientist systems that go beyond simply answering questions that we pose, and start to ask the kinds of questions that lead to open-ended creative discovery?

Tim Scarfe (0:15)

You spoke about Move 37, and I think you and I would agree that that was definitely creative.

Edward Hughes (0:20)

I don't think that Move 37 was creative. I think that Move 37 was innovative without being creative. What we find is that our Faraday agent is able to perform better than the frontier models, but it's also performing better than other frontier coding agents like Claude, for instance, and had these agents proactively reaching out to us, and trying to help us with stuff. To be perfectly honest with you, it was quite annoying. They just had no idea how to help us.

But at some point about two or three months ago, I think we reached a phase transition. What's the equivalent of OKRs for the age of recursive self-improvement? Indeed, and I think that is the next era. It's the era of collective intelligence rather than the era of individual intelligence.

Tim Scarfe (1:08)

This episode is supported by CyberFund.

If you're building at the frontier of AI, they want to hear from you. CyberFund believes the future belongs to AI natives who want to achieve the impossible. And that is why they're introducing the Monastery for AI native founders. It's an environment of pure focus and rapid execution for founders operating at AI native speed. And they're offering teams $2 million each to participate. Apply now at cyber.fund.

Just a quick piece on Inherent. So you were at Google DeepMind before. How much funding have you got? How many employees? What's the valuation?

Edward Hughes (1:54)

Yes. So we've raised $50 million from index ventures and radical ventures at a post-money valuation of $225 million.

Currently, we're a team of 11 people.

Tim Scarfe (2:08)

And presumably, you think that AI is the most important thing in the next five years, just generally.

Edward Hughes (2:15)

No, I wouldn't say that. I think that AI by itself, in some sense, is meaningless.

Really, it's what can AI enable in interaction with the wider ecosystem? And that for us, we're most interested in broader scientific progress.

Tim Scarfe (2:35)

Ed, it's amazing to have you on MLST. Welcome.

Edward Hughes (2:38)

Pleasure to be here.

Tim Scarfe (2:39)

Tell us about yourself.

Edward Hughes (2:41)

So, yes, my background. Originally, I was a theoretical physicist. That's my PhD. And I was studying string theory, in particular, the scattering of particles, but using string theory as a calculation mechanism. And one of the central theses of the work I was doing was an idea called dualities. In physics, a duality is a mathematical map, if you like, between two different theories. And it can take you in some quite unexpected places. So usually it's useful if you have calculations in one theory, which are very hard, you can map across this duality into calculations in another theory that are much easier. And so I was using some of these dualities in order to take calculations that were very difficult and map them into a different geometric space in order to do them. And I think that taught me a lot about how to think about transforming problems, something we'll probably come back to when we talk about creativity. But towards the end of my PhD, I became convinced that working in scattering amplitudes was not the most profitable thing I could do in terms of making an impact on the sum total of human knowledge. Funnily enough, many of the people that I was citing at the time of my PhD, people like Jared Kaplan and Jeffrey Pennington, also made a similar decision around about the same time.

So in 2016 towards the end of my PhD, I saw the AlphaGo match. I remember watching that, like so many other people being fascinated by Move 37 I became convinced that the future was going to involve agents that could really aid humans in making discovery and accelerate the rate of scientific progress. I joined DeepMind in 2017, really animated by that question, how do you build an agent that can itself make discoveries? I started by working on that in the context of reinforcement learning, particularly multi-agent reinforcement learning. The reason I was so interested in multi-agent reinforcement learning was because human technology seems to have been created not so much by one individual, but by the sum total of human culture. I like to think of cultural evolution as the fastest intelligence generating process in the universe. So I became, I suppose, obsessed by this idea of really trying to distill cultural evolution into agents. Now, at some point on my journey, the foundation model started to arise, and I started to work on building much larger models. I built a model called Adaptive Agents, which was all about using meta-reinforcement learning across a very large space of tasks in order to build. It was at that time, I think, the largest train from scratch RL agent. 500 million parameters, so it's pathetic by modern standards, but at the time it was rather large. Then I worked on world models. I suppose my claim to fame there was that I wrote by hand, effectively before coding agents, the final implementation of the GD1 model in a single file. I think that was the last time I really wrote a fully artisanal, human-written implementation of an algorithm.