Why Real-World AI Is Harder Than the Benchmark | EliseAI’s Mario Martone

Mario Martone (0:00)

I did not expect that, but even after coming from SYNC Theory, I would say the complexity of the problem of working now is probably higher now than it was before.

SPEAKER_2 (0:09)

An AI system that works 90 percent of the time might look impressive in a research paper, but when you're handling millions of conversations about people's housing and healthcare, that remaining 10 percent becomes an enormous problem. In this episode, EliseAI, Head of Applied Research, Mario Martone, joins MTS to explain what it takes to move AI from the lab into essential industries. Mario spent 15 years in theoretical physics, studying problems like black holes and string theory. Now, he applies that same research mindset to production AI, where his team has to decide when to use an LLM, when traditional machine learning is actually better, and how to evaluate systems operating across millions of real-world interactions.

They get into EliseAI's voice stack, its work fine-tuning open-source models, and how the company has automated workflows that can take someone from looking for an apartment to touring it and eventually moving in. And for Mario, the intellectual challenge hasn't gotten easier. He argues that some of the problems his team is tackling in production AI may be even more complex than the ones he encountered in theoretical physics.

Steph Smith (1:15)

Hello everyone and welcome back to MTS. Today, we are joined today by Doctor Doctor, we have two PhDs. So again, a double doctor, Mario Martone. He is the head of Applied Research at EliseAI, a company building conversational AI for essential industries like housing and healthcare. And he has a background in theoretical physics, physicist. He now leads a team applying the latest machine learning advancements to real world problems. So Mario, welcome to MTS.

Mario Martone (1:44)

Thank you. That's great.

Steph Smith (1:45)

Yeah. Welcome to the show. I did want to ask a little bit about your background because you started off being a professor and theoretical physicist, and now you're actually applying all of that learning into Applied AI.

And I was just sharing, we're having a little conversation right before we went live about why it's so important to use these skills and apply them especially to essential industries, like housing and healthcare are probably arguably some of the most essential industries. And so I wanted to just start off learning a little bit about you, why that changed, why EliseAI, and what did you take with you?

Mario Martone (2:21)

Right. Well, I think I agree with you that it's a big gap. So just to be precise, I mostly worked for 15 years on trying to understand the universe, black holes, string theory, things that are very, very kind of different from what I do daily on EliseAI. But I think there is something that actually is quite important, right? Like we are living in this moment where AI is moving so fast, and it's really unclear what the future will hold. And so there are extremely open questions. And so the similarity that I can see, while obviously working on string theory, things take forever, it's much slower than the pace. And here it's like, boom, boom, boom, every day there's like a change. But the easy question of like, the people who are able to like really think deeply about the problems can be very creative, can constantly kind of come up with new ideas, have a huge edge because AI is like a new field every day. So that's the thing that really, I think brought me here and allowed me to really have a big impact.

Steph Smith (3:19)

And tell me a little bit about the space. One of the big things, I mean, I'm a little bit biased because we just had Mina Song on last week at the New York Stock Exchange. We're talking all about essential industries and why it's so important to actually use this intelligence and get it diffused inside these industries. Just having stronger models on their own inside labs doesn't necessarily mean the impacts of that intelligence actually gets diffused into the real world, into actual industries.

Mario Martone (3:46)

In fact, for me, it was like, one other alternative I could have done, because I was kind of interested in going to industry, is to, for example, try to join one of these frontier labs, right? But the research that you do at frontier lab, it's kind of obviously different from theoretical physics, but it's much more alike. Like, you're working on this big problems, but one thing I really want it is to have an impact in the real world. And I think Elise is like such a unique company in that respect, because, you know, you can have these LLMs that are getting growing capabilities, but we are facing every day with the real challenges of what it means for having like, consequential decisions. So we actually, the decisions that the LLMs make for Elise for our products, they're not just like some random things that appears on the screen, but like drive impacts on people's life. And that could be very positive or very negative. So that's the thing that really excites me, and I think it makes Elise kind of such a unique space to be in.