The best model for your team? You haven’t invented it yet. | Ai2’s Tim Dettmers

**Andrew** (0:05)
Tim, I wanted to ask, just kind of like before we start diving into today, like I'm curious from a personal perspective, what is it like to sit in this kind of like swivel chair between academic and industry? Like what kind of perspective like has that been for you? And like, you know, what makes you most excited about being in that place?

**Tim Dettmers** (0:25)
Yeah, I mean, sort of industry and academia, the divide feels quite extreme.
That feels to be also sort of one divide sort of resources. I think there's also the divide in perspectives. And then it's also, if you look at locations like what's the thing that happens in the Bay Area, and it's just, if you're in a Bay Area, you're like in this bubble, everything's super exciting, everything goes super fast and that sort of thing. And some of it is true, but some of it is not. And so as an academic, you should sort of lean back and sort of take it in and sort of make up your own mind. And with that, you can actually carve out bits where you say, I can be competitive in this field, even if they have all the resources, but pursuing sort of particular ideas.
And that they might not be able to pursue. And so that is sort of an interesting space. But the other sort of is just this trade-off between working with the most resources and being sort of a cock on the wheel, or being sort of resource staff, but trying to make most of it. And, but yes, all the freedom. And you're working on big pieces that, you know, if I bring this out, there will be sort of my doing, instead of being like a tiny piece in a big team.

**Andrew** (1:43)
I love that framing that you prefer working in this almost like resource-strapped environment where you're able to fully explore the realm of thought and then not only explore it, but then forget the methodologies and get there. And whereas in this other environment where it's like resourceful and there's like so much that you can be pulling on and with each other, but the incentives are different. That's like it's a capitalism-minded machine. What you're trying to drive is a production and an output. And so like you said, the perspectives are so different. And you're after the quest of like, what is best? What are the foundational elements of this that we all need to understand? And that's like the really important, gritty work that maybe it's sometimes folks that are more industry-minded, like don't want to pay much attention to, right? And so it can be like a little difficult there.

**Tim Dettmers** (2:31)
Yeah. Maybe also a small story there that might be sort of quite interesting. And that is, doing my PhD, I was doing like sort of long-term internship at Meta. They had a lot of GPU resources. And it's like the dream of the PhD students had like hundreds of GPUs. So I had these hundreds of GPUs, and I was just running experiments, and experiments, and experiments. And I was making sort of progress, but not as much progress as I wanted to.
Then at some point, it was time to go back to the University of Washington and had much less resources. But now each experiment needs to be very carefully chosen, very carefully analyzed. And there I actually made some discoveries that I wouldn't have made if I just look at experimental results. Now I dove deep into the data, found some curious things, and I was actually more productive. So more resources doesn't necessarily mean better results. We can make more insights with less resources if you dive deeper.

**Andrew** (3:24)
In fact, when I think you have to be resourceful, that's when you're pushed to make those really ingenious kinds of discoveries. And it challenges all of us to strip away complexity instead of adding it on. It's a really great takeaway. And I'm really excited to explore all of this in our conversation. Because folks, today my guest on the show is Tim Dettmers, a research scientist at Ai2, and an assistant professor at Carnegie Mellon, who has built a career about finding the signal and the noise of high performance computing.
And like we just talked about, while larger companies and industries might have huge resources that cook up massive foundation models that can solve wide domain industry problems, Tim and his small team recently built Sarah, a state-of-the-art coding agent, through what he calls a hot plate and a frying pan. Like the complete opposite of being so full of those resources and working with a strappy team and strappy GPUs. And we're talking about the tactical engineering behind that today, the automation muscles that make it possible, but also the groundbreaking research that we've covered from Tim here on the show about how this type of breakthrough can allow more teams to return to and embrace specialized models of their own and not necessarily be beholden to foundational off the shelf tools. So, we have a big conversation to dive into today. It's very academic minded, super excited for it. And Tim, welcome to Dev Interrupted.