Scaling ChatGPT: Inside OpenAI's Rapid Growth and Technical Challenges | Evan Morikawa

**Evan Morikawa** (0:00)
It took a lot of tweaking to kind of find what the right utilization metrics were and then how to optimize those. And all of those are really critical to getting more out of it. But for us, everything was framed in terms of every improvement that we have represents more users we could let onto the platform.
As opposed to say like, oh, it's driving down our margins or making it faster. We kept cost and latency as relatively fixed. And the thing that could get to move was we got to get more users onto the system.

**SPEAKER_2** (0:32)
What's the impact of AI-generated code? How can you measure it?
Join LinearB and ThoughtWorks Global Head of AI Software Delivery as we explore the metrics that measure the impact GenAI has on the software delivery process in this first of its kind workshop. We'll share new data insights from our GenAI Impact Report, case studies into how teams are successfully leveraging GenAI, impact measurements including adoption, benefits and risk metrics, plus a live demo of how you can measure the impact of your GenAI initiative today. Register at linearb.io/events and join us January 25th, January 30th, or on demand to start measuring the ROI of your GenAI initiative today.

**Ben Lloyd Pearson** (1:15)
Hey everyone, welcome back to Dev Interrupted. I'm Ben Lloyd Pearson, Director of Developer Relations here at LinearB. I'm pleased to have Evan Morikawa joining us today.
Welcome to the show, Evan.

**Evan Morikawa** (1:27)
Awesome, thank you.

**Ben Lloyd Pearson** (1:28)
Yeah, so full disclosure, Evan and I worked together in a past life at an API company. You may not remember this, but you were actually a big part of why I decided to join that company.

**Evan Morikawa** (1:38)
I did not know that.

**Ben Lloyd Pearson** (1:39)
Yeah, none. You saw your video on YouTube.

**Evan Morikawa** (1:42)
Oh wow.

**Ben Lloyd Pearson** (1:44)
So yeah, we've got a little bit of a history, so it's really great to just have an opportunity to catch up with you, talk about some of the new stuff that you're working on. I know personally, I was a little bit envious when I saw that we were going to OpenAPI, because I was like, that sounds so cool. And here we are talking about how cool your work is. Let's kick it off. I mean, OpenAI, I feel like needs no introduction. I mean, I feel like everyone is talking about it. It's gone viral. At the center of the conversation around AI and LLMs, the release of Chat-GPT has kicked off a global phenomenon. And I want to walk through that story, in particular, what it took for you to scale Chat-GPT when you had that viral moment.
And especially, I know I've heard a little bit that there was a shortage of GPUs that also affected this. I do want to dig into that. And I just want to learn a little bit about how you've been flexible and nimble while this industry has just rapidly shifted around you.
So, let's just kick it off. Where does this story all start for you? Like, how did you get to where you are right now?

**Evan Morikawa** (2:44)
Yeah, so, I mean, as you mentioned, I was working at an API company before.
And at the time, that's all OpenAI had. In fact, that was the light bulb moment. Was learning that three years ago or so now, OpenAI was starting this Applied team. And Applied was all about bringing this crazy technology safely to market. And at the time, that was just a single developer-facing API around the very first GPT-3 models. Which is good, because up until that, it's still the case that I do not have machine learning background. OpenAI was much more research lab focused. But then here now, it's starting up this brand new small team doing APIs and products. It was like, oh, I've been doing APIs and products. Maybe I can help contribute here. I think there was also that feeling, too, that I feel like I had a reasonable sense of how computers worked by that point in my career, except for this. This still is this kind of magic.
Certainly at the beginning was this magic black box. I was like, okay, I'm gonna see what that's all about as well. I also knew of some of the founders through just the broader network, and yeah, we reached out and that was the start of it there. When I joined, Applied was very small. There were only basically half a dozen engineers working on all of the APIs and systems for that. And that steadily grew as we were trying to work on iterating on these language models. I think the next big, the big really first release or push of any kind was when we tweaked these to write code and worked with GitHub to launch GitHub Copilot. So GitHub Copilot actually initially ran through our servers at launch because it was very difficult to run or deploy these. That was definitely the first time we had any experience running this at any sort of scale.