Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk

Shawn Wang (0:04)

We're here at Periodic. We're here today with Liam and Doğuş from Periodic. Welcome and thanks for having us at yours.

Liam Fedus (0:10)

Yeah, great to be here and thanks for coming in.

Shawn Wang (0:14)

I want to start off with one of these quotes that I love from your website. It says, intelligence is necessary but not sufficient. New knowledge is created when ideas are found to be consistent with reality.

Seems so straightforward, but why is it non-obvious?

Liam Fedus (0:29)

I think that's the thesis and premise behind Periodic that Doğuş and I thought when we were building this, which is you can't just think your way to a solution.

The universe is so complicated that in order to actually push the frontier of knowledge and to make progress, you need to create these conjectures and then actually see whether or not it holds. No amount of rereading that textbook or paper and just thinking, like disappearing into a room, is going to allow you to think through all possible experimental outcomes, expected and unexpected, and you really need this iterative process. That's our core belief for putting this together, and that's why from the very beginning, we need to have the AI systems, the simulations of the physical world, but then also to build up the physical high-throughput experiment. We think a different type of intelligence emerges from that.

Shawn Wang (1:21)

I think it's also notable with OpenAI and Google, you don't have that resources in those big labs. You had to come out and do your own thing, because you are inventing your own playbook as you go along.

Ekin Dogus Cubuk (1:33)

Yeah. A team like this has never existed, I think. We try to bring together solid-state chemists, solid-state physicists, experimentalists, theorists, hardware engineers, LLM experts, computer scientists.

Some of these technologies are very recent, like the high-throughput experimentation, the robotic arms have only been tried in the last three years or so. The force field expertise, some of these force fields are also very recent. But yeah, we felt like having a lab is very important, having this focus and bringing all these people together to work together is very important.

I guess the closest examples from history are places like Bell Labs, where incredible theorists and experimentalists and chemists work together and achieved incredible things. So we're trying to do the same here.

Brandon (2:17)

I want to talk about force fields in a bit. But before we get to that, yeah, like what are they? But before we get to that, I think one of the, just for context, one of the biggest differences between, you know, some of our audience are AI, you know, engineers, and some of the audience is scientists. So let's go and talk to the engineers real quick and say, like, what is the difference if you've been working at OpenAI as a big lab? What's the difference between what you do there and what is, you know, what is here? I think you've already hinted at this, but maybe explicitly. What's the, how do you have to reframe your thought process?

Liam Fedus (2:49)

Well, I mean, I think, like, one interesting thing is now our reinforcement learning environments literally derive from the environment, from our physical labs.

Our data comes from our physical labs, and this is sort of our ultimate truth. It's not enough just to do optimization against some answers that were known in some papers or textbooks because we're going beyond that. And I think that's one of the biggest differences. But along with that, there's issues of, say, decision-making under uncertainty. So when you're doing optimization against math, there's a high precision to it. You're not really dealing with variance or uncertainties or aberrant measurements, whereas that's very key to our process.

For example, when we are doing materials discovery loop, things don't come out of the furnace labeled, right? Even the labeling process can be stochastic and noisy. And sometimes you'll have aberrations between different machines.

Maybe we'll have insufficient telemetry. So we thought we ran it at this temperature, but really the temperature was some delta away. And an intelligence that can take in this noisy data and make intelligent decisions like a scientist would is a different set of reasoning strategies. So I think there's a huge amount of commonalities. So the standard mid-training, reinforcement learning, the construction of tool-using agents, variance reduction, making sure infrared doesn't have mismatches between training and inference. But we have to go beyond that and really think about, how do you do accurate work when there's a high amount of uncertainty? How do you make really incredibly efficient use of a limited set of data? So we're scaling things up significantly, but still it's very different than a digital environment, where you can arbitrarily add more environments or more rollouts. You don't have that same capacity. So sample efficiency is another key thing.