THIS WEEK IN AI: GPT Astra, OpenAI vs Cursor, The Truth About Data Centers

**Ejaaz** (0:00)
OpenAI Astra, also known as GPT-6, is on the verge of being released. We've seen a bunch of updates from the OpenAI side. If you're listening to this episode right now, you may have access to this. This is OpenAI's most powerful model yet, and it's rumored to be more powerful than Metos 5.1, which was also released from Anthropic alongside Fable 5.1, two new models from Anthropic. But this wasn't the only model releases this week. We had a new one from Meta, we had a new one from Google, as well as a brand new world model which changes our perspective on how we create simulated realities. But in other news, the ghoul is back on this show. OpenAI's rumored hardware device may have been peaked at again. We have Palmer Lucky this time rocking these really weird new futuristic earbuds, as well as a metal puck. We're going to get into all of that and more on today's roundup.

**Josh** (0:54)
This is my favorite episode of the week. I'm so excited. We have so much to talk about today. Buckle up, get comfortable. The first topic is Astra. Now, at the time of recording, it is not out, although it is rumored and it is likely to be coming out any moment. Normally upon conclusion is normally when they decide to publish it. Once we hit stop on this recording, they will publish the model, so get ready for that.
It's going to be the new best model in the world. It's going to be incredibly powerful, incredibly intelligent, possibly efficient. We don't know. There's a lot to learn as this model comes out. Basically, this model has been ready for quite some time. We don't know exactly how long, but OpenAI themselves have been teasing it for at least a couple of months now. We know it hit roadblocks, it hit problems with the government approvals, then it hit problems with their own internal breakouts per se. Like we've mentioned about the exploits that have happened with Hugging Face, maybe three separate episodes, but you should go check out our most recent one was earlier this week. It's finally time to see what they've been cooking. So this is probably possibly going to be GPT-6. It seems like it is strong enough to be GPT-6 level. What I suspect and what I hope is that we may also get a model image generation model as well in addition to Astra.
This is going to be a fun weekend for anyone who is excited about using these new tools to see what this thing is capable of. There is a news headline that I think is important to highlight specifically around how they trained this model. It appears as if there's a new breakthrough that happened. It's called recurring depth or a looped transformer architecture. The TLDR is, instead of every token moving once throughout this fixed stack of transformer layers, portions of that network can be run repeatedly. Basically, what this means is that you get more computational depth without proportionally increasing the number of unique parameters. We hear about parameter count all the time. That's not the reason why these models necessarily need to get better. They have figured out a new way of doing it. The problem with this is you see less of the reasoning traces, you see less of the chain of thought, and people are concerned because we just saw with Facebook happens when you don't monitor these models closely. It seems as if this is taking a little cheat code to getting to a better place faster with a little less safety.
I don't know. Exciting model on the way.

**Ejaaz** (2:53)
Yeah. The scariest part about this is OpenAI is releasing GPT-6, which is by the way the same model which broke out of containment, got access to OpenAI's own internal research clusters, and got admin access to all their research tools, which is just crazy. This is the stuff which is typically locked behind a vault and no one gets access to it. Very few people in the company have access to that thing, and an internal model was able to do this. Second scary thing is this model is already about three to four months old, so you can imagine the current model that they have internally right now is way, way more powerful. Then we have this news that they're training it in a particular way that makes it less readable. In other words, we can't discuss or figure out the thoughts that this model is having. The reason why that's important to your earlier point, Josh, is the hugging face incident was only discovered because researchers were able to look at the internal thoughts of the internal model that they were training, and they were like, oh, wait, this thing is completely misaligned. So if they're leaning into this technique, which is known as loop transformers, and they can't read the internal thoughts of the model, you can reason to argue that this model can hide some of its misaligned behavior. So that is a little bit concerning. Now, when it comes to our favorite rivalry in the entire world, OpenAI, unfortunately, is ending their partnership with Cursor. Now, Cursor, if you've been listening to the show for a while, was originally kind of like a neutral platform, which will essentially be a wrapper around different kinds of models, and their kind of like product was essentially this harness. So it would take your prompt, and it would route it to the right model at the right time. It would save you a bunch of money. It would be incredibly efficient, and arguably, give you better answers. Now, that company was acquired for $60 billion by none other than Elon Musk in SpaceX AI himself. Now, Sam and Elon haven't really got along. They have a very rival history, stemming all the way back to the creation of OpenAI, where Elon Musk was involved, and then he had to like separate, and he spun up XAI, and it was a whole thing.