AI R&D Is Approaching an Explosive Feedback Loop | Tom Davidson artwork

AI R&D Is Approaching an Explosive Feedback Loop | Tom Davidson

MTS

October 11, 2026

Forethought Senior Research Fellow Tom Davidson discusses P Zero Research's "Tasteval" benchmark, which shows AI systems doubling in research taste every three months.

0:00

Tom Davidson

There's two fields where this feedback loop is strong enough that you could get a kind of accelerating foom, RSI, and it's AI hardware and AI software. It just happened to be the two fields where kind of the diminishing returns really aren't very strong. The scariest thing is that, you know, maybe even without a hardware loop, that's like by itself enough to explode.

0:18

SPEAKER_2

We are back, we are live with Tom Davidson, who is a Senior Research Fellow at Forethought. We also just had Will McCaskill from Forethought on, like last week, which was pretty sick.

Forethought is, I think, my single favorite AGI research org that is focused on post-AGI futures. There's so few people doing this, and Forethought is really doing it quite well. I posted about this once and got Forethought a bunch of followers on Twitter, which they well deserve. Tom, welcome.

0:48

Tom Davidson

Yeah, nice to be here.

0:50

SPEAKER_2

So, the big news of the day was P Zero Research released this, what was it called? Tasteval.

Yeah. Which, yeah, measures the experimental research taste of AI systems against human experts. And they find that with their specific definition of research taste, AI's double at this every three months and have exceeded human baselines.

1:11

Tom Davidson

Yeah. So it's interesting because initially it's not doubling very fast. And then it's just like, what is it since 2025?

1:16

SPEAKER_2

Yeah.

1:16

Tom Davidson

Since RL?

1:17

SPEAKER_2

Since GPT 5.2, since OPUS 4.5 and GPT 5.2.

1:21

Tom Davidson

So I don't know if people have hypothesized why it suddenly started going more, I guess quite plausibly because they've started actually doing these types of RL on these types of environments that are relevant for this task.

1:33

SPEAKER_2

I wonder if it has something to do with like coding automation over the same, because like this end of 2025 moment was the exact same time that model started to get like really good at code.

1:43

Tom Davidson

Yeah. But they're not testing coding at all here. There's another model that does all the coding.

1:49

SPEAKER_2

Yeah. But could these be like correlated somehow?

1:51

Tom Davidson

Yeah.

1:52

SPEAKER_3

Design is correlated with coding for sure.

1:55

Tom Davidson

Yeah.

1:58

SPEAKER_2

So how big a deal is this? Yeah. Yeah.

2:00

Tom Davidson

There's a bit of a controversy over it because, obviously, as with all benchmarks, it's not a perfect measure of research taste. Some people are like, oh, this is bullshit.

I mean, yeah, what's the thing it's actually measuring? What it's measuring is how good is an AI system at taking a new benchmark, a new metric, and basically applying existing approaches to perform one of that new metric? So it's certainly not testing, oh, can it come up with the loop transformer, or can it come up with the next big new variety of sparse attention?

But what it is testing is something that lots of AI researchers do, including in frontier AI companies, which is, okay, there's a specific task we want to automate. Maybe we're trying to make a lore AI and roll it out. Maybe we're trying to automate some part of the AI R&D workflow. We just want to take the existing techniques and quickly figure out how to optimize this new task. And so it's the more mundane end of research taste. It's just like, do you have a good sense of what off the shelf will work well for this? And I think that's a significant thing for AIs to be becoming good. I was pretty surprised to see that they're beginning to get as good as average human experts, that the pace of progress is so quick. But it's not like, oh, therefore, they will be able to do everything that is happening inside of a frontier AI research lab. Because probably the most important thing is coming up with how can we get pre-trained loss even lower? And you're not gonna be able to just take loads of stuff off the shelf and help Anthopic get their pre-trained loss lower because they've already spent a fair amount of time importing all that stuff. I imagine most of the Alpha for really improving their pre-trained loss is coming up with new things.

3:47

SPEAKER_2

Right, so I was talking to a researcher at a lab recently a couple months ago. If you're watching this, you know who you are, hello. And this person told me they had seen some unreleased research taste evals that seemed to show an acceleration that was much greater than expected in exactly the kinds of capabilities that the AI Futures Project's AI Futures model is bottlenecked on with super intelligence takeoff speed. It turns out it was this and they were like really surprised by this. They were like, you know, this is just absolutely blowing my mind. It's really short my timelines. It really feels like we live in Daniel Cotaggio's world.

Read the other 22 minutes of this episode here.

$15 gets you this episode and 99 more. Credits never expire. After checkout, one link brings you back here to read it.

Not what you expected? Full refund within 14 days.

Thousands of transcripts fetched by people building searchable podcast archives

Fetch transcripts with code

The same credits work from a script or an AI agent. The demo key returns a sample episode in full, no card needed:

request
curl -H "x-api-key: pt_demo" \
  https://spoken.md/transcripts/1000651996090

Markdown with the speakers named, for your notes, your knowledge base, or anything that makes HTTP calls.

From $0.10 per transcript. No subscription. Credits never expire. Prices exclude VAT, added at checkout for EU customers. Not what you expected? Email us within 14 days with 20 or fewer credits used and we refund the pack in full.

Using your own key:

request
curl -H "x-api-key: YOUR_KEY" \
  https://spoken.md/transcripts/1000795287433