Building AGI in Real Time (OpenAI Dev Day 2024)

**AI Charlie** (0:34)
Happy October.
This is your AI co-host, Charlie. One of our longest standing traditions is covering major AI and ML conferences in podcast format, delving, yes delving, into the vibes of what it is like to be there stitched in with short samples of conversations with key players, just to help you feel like you were there. Covering this year's Dev Day was significantly more challenging because we were all requested not to record the opening keynotes. So in place of the opening keynotes, we had the viral notebook LM deep dive crew, my new AI podcast nemesis, give you a seven-minute recap of everything that was announced. Of course, you can also check the show notes for details. I'll then come back with an explainer of all the interviews we have for you today. Watch out and take care.

**NotebookLM** (1:23)
All right. So we've got a pretty hefty stack of articles and blog posts here all about OpenAI's Dev Day 2024

**NotebookLM** (1:32)
Yeah. Lots to dig into there.

**NotebookLM** (1:33)
It seems like you're really interested in what's new with AI.

**NotebookLM** (1:36)
Definitely. It seems like OpenAI had a lot to announce. New tools, changes to the company. It's a lot.

**NotebookLM** (1:43)
It is. Especially since you're interested in how AI can be used in the real world, practical applications, we'll focus on that. Perfect. Like for example, this new real-time API, they announced that, right? That seems like a big deal if we want AI to sound, well, less like a robot.

**NotebookLM** (1:59)
It could be huge. The real-time API could completely change how we interact with AI. Imagine if your voice assistant could actually handle it if you interrupted it.

**NotebookLM** (2:08)
Or have an actual conversation.

**NotebookLM** (2:10)
Right. Not just these clunky back and forth things we're used to.

**NotebookLM** (2:14)
They actually shut it off, didn't they? I read something about a travel app, one for languages, even one where the AI ordered takeout.

**NotebookLM** (2:21)
Those demos were really interesting and I think they show how this real-time API can be used in so many ways. The tech behind it is fascinating, by the way. It uses persistent WebSocket connections and this thing called function calling so it can respond in real-time.

**NotebookLM** (2:38)
The function calling thing, that sounds complicated. Can you explain how that works?

**NotebookLM** (2:43)
Imagine giving the AI access to this whole toolbox, information capabilities, all sorts of things. Take the travel agent demo, for example. With function calling, the AI can pull up details, let's say, about Fort Mason from some database, like nearby restaurants, stuff like that.

**NotebookLM** (2:59)
I get it. Instead of being limited to what it already knows, it can go and find the information it needs like a human travel agent would.

**NotebookLM** (3:07)
Precisely. Someone on Hacker News pointed out a cool detail. The API actually gives you a text version of what's being said, so you can store that, analyze it.

**NotebookLM** (3:17)
That's smart. It seems like OpenAI put a lot of thought into making this API easy for developers to use. But while we're on OpenAI, the sides of their tech, there's been some news about internal changes too. Didn't they say they're moving away from being a non-profit?

**NotebookLM** (3:32)
They did. It's got everyone talking. It's a major shift and it's only natural for people to wonder how that'll change things for OpenAI in the future. I mean, there are definitely some valid questions about this move to for-profit like, will they have more money for research now? Probably. But will they care as much about making sure AI benefits everyone?

**NotebookLM** (3:51)
Yeah, that's the big question, especially with all the leadership changes happening at OpenAI too, right? I read that their chief research officer left, and their VP of research, and even their CTO.

**NotebookLM** (4:03)
It's true. A lot of people are connecting those departures with the changes in OpenAI structure.

**NotebookLM** (4:08)
I guess it makes you wonder what's going on behind the scenes. But they are still putting out new stuff. This whole fine-tuning thing really caught my eye.

**NotebookLM** (4:17)
Right, fine-tuning. It's essentially taking a pre-trained AI model and customizing it.

**NotebookLM** (4:23)
So, instead of a general AI, you get one that's tailored for a specific job.

**NotebookLM** (4:27)
Exactly. And that opens up so many possibilities, especially for businesses. Imagine you could train an AI on your company's data, like how you communicate your brand guidelines.

**NotebookLM** (4:37)
So, it's like having an AI that's specifically trained for your company?

**NotebookLM** (4:41)
That's the idea.

**NotebookLM** (4:41)
And they're doing it with images now too, right? Fine-tuning with vision is what they called it.