90% of AI prototypes never reach production (w/ Temporal's Samar Abbas) | AI Basics

Jason Calacanis (0:00)

All right, everybody, welcome back to our basic series. What is our basic series? Well, on This Week in Startups, we get asked the same basic questions over and over again. Sometimes it's a legal question for founders, sometimes it's finance, sometimes it's about product market fit or getting customers, and increasingly, and almost all the time, it's about basic things in AI.

Every startup is not just building AI products, but they're using AI to build those products, right? It's, you could only be competitive today if you're using AI inside your startup. So today on AI Basics, our guest is Samar Abbas. He is the CEO of Temporal, temporal.io, if you want to go take a look at the website. And he's a 20-year engineering veteran, worked at AWS, Microsoft and Uber. Today, we're going to talk about why agents desperately need harnesses and how enterprises can safely dip their toes into the agentic era and why he's building Temporal on Google Cloud. Welcome, Samar.

Samar Abbas (1:01)

Thanks, Jason, for having me here. Excited for the conversation.

Jason Calacanis (1:04)

Yeah, and so you've been at this for a while. You've been an engineer for a while. How have things, just as a warm up here, how have things changed from when we were building products and services 10 years ago to how products and services have been built in the last 10 months or even the last 10 weeks? Because it feels like it's changed even between those two timeframes, yeah?

Samar Abbas (1:26)

Yeah, so Jason, like I get this question a lot, like a lot, everything looks different, especially in this world of there's clearly a new platform shift happening with the rise of AI. Essentially, these models get smarter and smarter.

One of the things that I see a lot is although the way people are thinking about applications is very different, like especially with the rise of now coding agents coming in the mix, everyone suddenly is a software developer on the planet. Now because software is now more approachable to a wider class of people out there, not just like people with computer science backgrounds, like everyone can build up. If you have an idea, coding agents are making those more accessible to everyone on the planet. So on one side, it feels like the way application, the entire new application platform is emerging on one end. But on the other side, what I see is we are seeing a very similar set of problems. That's why you hear a lot that, oh, I have an awesome idea. I got started and built an application through a coding agent. But 90 percent of those ideas die after a POC, essentially. They never see light of the day. Why is that?

Jason Calacanis (2:42)

They die at that proof of concept stage. People get super excited. But then all of a sudden, there's this despair. It's not stable, it's brittle, it's not replicable. What's the series of problems here? Because I see that all the time. People get excited inside my organization. They show me something and then I'm like, okay, is it ready for production? And the answer is inevitably, I don't know. It doesn't feel like it, but maybe.

And that's really the difference between production level software and a proof of concept. There's a big gap there, isn't there?

Samar Abbas (3:12)

There is a big gap there. And I think this is where the worlds are starting to look more and more similar to what we are seeing in this platform shift, essentially, is as more and more of AI applications, and especially these agents, are starting to hit production.

What we are seeing is they get longer and longer durations in time. The more scale, the more asynchronous they get, the more longer they run, the more actions they take in real world, the more value they create. It's so clear for every organization that that's the pattern. But the moment they go there, you start running into the same set of reliability, scalability, durability challenges, and for such a level, which feels very similar to when the Cloud Platform shift happening. Because suddenly, all of these agents are now starting to graduate from running from your laptops into a real distributed environments, and these problems starts to look more and more similar to distributed in nature rather than a completely new AI problem essentially.

Jason Calacanis (4:16)

And one of the problems is the durability of the execution.

Everybody's had the experience, no matter what AI product you're using, that it's working, it's working, it's working, and then it peters out, it stops, and you have to start over. You might have to start over a very large job. This happens in generative AI, whether you're making videos or images or sounds. This happens when you're doing research reports or you're doing big web research, whatever the product or service you're building is, durability keeps happening, and then it doesn't seem to know, because this is not perfect software yet, and these large language models in some cases are making their best prediction, it doesn't know where it left off. Why is that, and how do you solve that problem, this durability when running?