Emergency Pod: OpenAI's new Functions API, 75% Price Drop, 4x Context Length (w/ Alex Volkov, Simon Willison, Riley Goodside, Joshua Lochner, Stefania Druga, Eric Elliott, Mayo Oshin et al)

**Swix** (0:05)
Hello, everyone. This is Swix, back again with another emergency pod. The last time we did this was in March when OpenAI released Chat2BT plugins, and the new functions API today is effectively the Chat2BT plugins API, now available to all developers, and with a whole bunch of other news, 75% price drops on embeddings, four times the context length, and a lot more other updates. So what we do in these situations, when there's breaking news, and it's very developer focus, is we convene all the friends of the pod, with Simon Willison, Riley Goodside, with people from Microsoft Research, Hugging Face, and Pinecone, and more that I don't even know where they work at. I think some of them used to also contribute to Langchain.
But anyway, we just had all our developer friends, we had 1,400 people tune in yesterday, just to talk about what they think, and what they want to build with the new functions API.
We aim for Latent Space, we're very much targeting for Latent Space to be the first place that people hear about developer-relevant news and to go deep on technical details to think about what they can build with them, and to hear rumors and news about anything and everything that they can build with. So enjoy. Unfortunately, Alessio was on vacation, so he couldn't help co-hosts, but fortunately, friend of the pod, Alex, joined in and that's going to be the first voice that you hear. Alex has been doing a fantastic job running Twitter spaces every Thursday. If you want to talk about just general AI stuff, as well as just follow him for his recaps of really great news. So without any further ado, here is our discussion on OpenAI's Functions API and the rest of the June updates.

**Alex Volkov** (1:47)
For those of you who work with OpenAI 3.5 and 4, et cetera, feel free to raise your hand and come up, ask questions as we explore this together.
And I'll just say thanks to a few folks who joined me on stage, Niston and Jon.
And we've been doing some of these updates every Thursday, but this one is an emergency session. So we'll see, maybe Thursday we'll cover some more. So OpenAI today released an update, the June update with a bunch of stuff.
And we'll start with the simple ones, but we're here to discuss kind of the developer things. We'll start with the pricing updates. So 75% reduction in embedding price. This follows a 90% reduction of embedding costs back in, I want to say November, December. Anybody remember that? Maybe Roie in the audience. Roie, feel free to come up as well.
And we've seen kind of this reduction in cost on a, on a trajectory to basically, you know, being able to embed the whole internet. There's actually, I want to find this, there's actually a tweet by Boris from OpenAI that talks about approximately it's going to cost you $50 million to embed the whole internet. Like all of the text on the internet pretty much. And Logan followed up today and said, you know, after the updates of the pricing today, it's around $12.5 million versus $50 million before. Just to give like a huge scope of numbers in terms of like, how fast this type of tech advances.
And we have Zenova in the audience. Zenova, feel free when you finish eating. But basically there's now a debate whether or not embedding on client side is worth it, given that it's like so, so, so free or almost like very, very cheap to embed stuff. Obviously it's an API and there's concerns about using private data, but embedding is 75% cheaper. Imagine that you run embedding in production. And Jon, let me know if you do, Houston. Today, if you switch to this API, you basically just received a 75% like price cut for, for the use of a bunch of, a bunch of stuff. And by the Riley from Dexa here, once he joins, they also, they do a bunch of embeddings for pretty much every podcast out there. So, you know, just in one day, you can receive like a significant, significant decrease in costs. So embedding price goes down significantly. Very exciting. In addition to this, another price cut is the 25% for GPT 3.5.

**Joshua Lochner** (4:11)
Yeah, I just want to say, so we use, we use a lot of embeddings. We're really happy to see that. But the thing I'm most excited about pricing wise today, it's the new 16K GPT 3.5 model, because I believe it's about 150% the price of GPT 3.5 trouble previous API pricing, or what it is now rather. And this is really significant for us because GPT 3.5 has never had that many tokens that you can max out in your context window. So when we build our input prompts for our copilots, it's usually using most of the 4K window just for the input.