AI Cannot Invent Good Hypotheses | Greg Kamradt
MTS
October 28, 2026
ARC Prize Foundation President Greg Kamradt explains why OpenAI's Astra crushed ARC-AGI-3 with 99% accuracy and why current LLM scaling hits a wall before real AGI. He breaks down why open-ended discovery remains uniquely human and what the next generation of ungameable AI benchmarks must look like.
Greg Kamradt
The role that humans play, and in general, if you think about how you use AI right now, it's very good at executing what you want it to do. It's not very good at coming up with ideas, right? It's not very good at coming up with hypotheses. We tried to make V3 games with AI-generated ideas, and they were all crap, they were no good.
They lacked the special substance about what we wanted to make a good V3 game.
SPEAKER_2
We're back, we're live with Greg Kamradt, who is the president of the ARC Prize Foundation, which develops benchmarks and competitions to measure progress toward more adaptable AI. Something like real AGI, whatever real AGI means. Most famously, Arc-Agi.
Greg, welcome to MTS.
Greg Kamradt
Yeah, thank you very much for having me today.
SPEAKER_2
I'm sure we're all dying to know, what should we start with? What's next after Arc-Agi-3?
Greg Kamradt
Wow.
Well, the vibe that's going on right now is, so benchmarks are getting harder, but it's in our opinion that we need categorically different benchmarks. So the old paradigm, just question and answer, is not what is going to come up next. So you'll see other benchmarks come out there. Most recently, something like Program Bench, where you show the AI a different type of program and it tries to reverse engineer what that program is. Benchmarks are leaning a lot more into creativity and actually having the AI produce something as opposed to just answering a question.
SPEAKER_3
Yeah. So our games, I'm curious, do you see them as the optimal balance of something that requires some sort of creativity, but also tactical problem solving?
SPEAKER_2
Sure.
Greg Kamradt
Well, I mean, if you think about a game, another word for game is environment, right?
SPEAKER_2
Yeah.
Greg Kamradt
And if you think about the ultimate environment, it's physics. It's the reality that we live in. So games as an environment are quite interesting, but what's really cool about games is there's a spectrum of complexity in the environment, a spectrum of open-endedness in the environment. So you can have a game that's super simple like Tic-Tac-Toe, or have a game that's very complicated like StarCraft or something like that. And so games go deep, but the ultimate environment is physics and reality that comes from there. So it's a big spectrum.
SPEAKER_3
Yeah.
Greg Kamradt
Yeah.
SPEAKER_3
Yeah. I'm curious, what are some of the games that we obviously know about, like AlphaGo and AI has a long history with games, but what categorizes games that are easy, furry, primitive or less intelligent model versus harder?
Greg Kamradt
So basically, what are the games that show you the spectrum of intelligence?
SPEAKER_3
Yes. Yes.
Greg Kamradt
What is interesting is, so ARC is an organization, we're a non-profit and we build benchmarks to try to show the gap between what humans can do and what AI can do. Because our assertion is that humans are only proof point of general intelligence, and we assert that we do not yet have AGI. So therefore, there must be a gap between the humans can do and what AI can do.
What we are honing in on and what is still human-specific, is open-ended discovery, the scientific method, coming up with good questions, coming up with hypotheses. So that is very, very human-esque. And so what we are looking towards is environments that elicit that type of response from AI. Because if it's not able to do that, then that shows that there's clearly a gap for it, and we do not yet have AGI.
SPEAKER_2
Well, how does something like Arc-Agi indicate that the model is better at scientific discovery?
Greg Kamradt
Yeah, the Arc-Agi series starts narrow and expands slowly and slowly goes out from there. If you look at Arc-1 and Arc-2, those are simple JSON grids. Just predict the transformation, input and output.
Arc-3 was a little bit wider because we didn't tell you what the actual goal was. As we think about Arc-4, Arc-5, Arc-6, the environments just need to get wider and wider and wider and elicit more creativity and a wider search space for models to go after.
SPEAKER_3
Right. So why was Astra particularly good at Arc-Agi-3?
Greg Kamradt
Yeah.
Astra was very good at Arc-Agi-3 using one of their open-air harnesses. I think it got something like 99 percent. So more or less it saturated the benchmark. The reason why is because Arc-Agi-3 games had a finite ceiling. There was one answer to go and do, and you didn't have to produce anything along the way. All you had to do is just go and solve the game. Each one of the different games were simple.
Read the other 15 minutes of this episode here.
$15 gets you this episode and 99 more. Credits never expire. After checkout, one link brings you back here to read it.
Not what you expected? Full refund within 14 days.
Thousands of transcripts fetched by people building searchable podcast archives
Fetch transcripts with code
The same credits work from a script or an AI agent. The demo key returns a sample episode in full, no card needed:
curl -H "x-api-key: pt_demo" \
https://spoken.md/transcripts/1000651996090Markdown with the speakers named, for your notes, your knowledge base, or anything that makes HTTP calls.
From $0.10 per transcript. No subscription. Credits never expire. Prices exclude VAT, added at checkout for EU customers. Not what you expected? Email us within 14 days with 20 or fewer credits used and we refund the pack in full.
Using your own key:
curl -H "x-api-key: YOUR_KEY" \
https://spoken.md/transcripts/1000798450000