#256 - Fable 5.1, Astra Tease, Gemini 3.8 Flash

Andrey Kurenkov (0:11)

Hello, and welcome to Last Week in AI podcast. We can hear a chat about what's going on with AI.

As usual, in this episode, we will summarize and discuss some of last week's most interesting AI news. I am one of your regular hosts, Andrey Kurenkov. I studied AI in grad school and now work at the startup AstroKate.

Jeremie Harris (0:29)

Hi, everybody. I'm your other regular co-host, Jeremie Harris from Claude Stone AI. I do AI national security stuff, AI loss of control stuff. I guess now we're calling that rogue AI stuff in US, China, et cetera, et cetera. You know the drill.

Andrey Kurenkov (0:41)

Much more exciting. We're just seeing AI safety. That's right. Rogue AI.

Jeremie Harris (0:45)

That's right.

The funny thing is for years at the level of introducing myself, like the first time that somebody hears what I'm doing, I'd be like, yes, I work on either AI safety or I would say sometimes AI security, because shortly after the Trump Admin started, safety became a bad word the world over. This includes the UK, by the way. So like Keira Starmer was very down on it, and it became the AI Security Institute. Everything was AI security. And now I'm kind of like, well, I've always obviously talked about loss of control very directly. But at that level of introducing myself, I find that's actually changed. And I think that's really cool because that did reflect the massive tax that everybody in the space, really everybody, ended up paying for even being seen to talk about these things.

Andrey Kurenkov (1:31)

Including, I think, in the AI community, outside of the AI safety community in and of itself, despite even like Neurobes, for instance, big conferences, requiring people to do things like impact statements, which would imply that safety is a concern, and we do think it will be impacts on society, but for quite a while, safety was a marginal kind of perspective, let's say, or even looked down upon. And now I think, certainly with this recent ChatGPT story, which we'll be talking a little bit more about, it's not only become known, it's kind of mainstream now to be aware that this is a real thing.

Jeremie Harris (2:13)

Yeah, and even the word safety, right? People started to use it, and in my opinion, they kind of mangle it by making it refer to more prosaic sort of weaponization. Weaponization is a really serious concern. A lot of our work is in that direction, for sure. But it really originally meant AI alignment, and even AI alignment got watered down to the point where we had to invent super alignment. And now even the word super intelligence is getting co-opted. So there's this thing in the history of language where people will invent a word to describe people, for example, with now we might say mental disabilities a few years back. You might use a different word that started with R and that probably gets us flagged or something. But that used to be a completely unobjectionable medical term, right? And so what's going to happen is people are going to shift over to the new thing. Oh, the people already say stuff like, oh, you're special, you're like special needs or whatever is already becoming that. And we've had that kind of happen in reverse where like the gradual encroachment, I guess the incentive is always to, for the labs especially, to interpret alignment and loss of control in the way that is easiest for them to actually deal with. And because no one has a clue how to control super intelligence, that's meant that they have chosen to continually interpret AI safety as more prosaic sort of let's not make it say bad words. And then it was like, AI alignment became that and like, you know, so I think a lot of the terminology got muddled in that way.

And now we're sort of finally at the point where there's a thing we can point to, to be like, no, the hugging face thing. Like, what are you doing about that for the next generation of models up to super intelligence?

Andrey Kurenkov (3:47)

It's very interesting now, like that lay people, so to speak, even know about the hugging face thing, which usually these kinds of things don't get out there. We'd like to thank Box for being a sponsor. Box is building the intelligent content management platform for the AI era. It access the secure content foundation, where AI agents don't just access your unique institutional knowledge, they actually orchestrate end-to-end workflows across your business critical systems. And that's important. If you're still using AI to just do basic chat and summarization, you're probably not getting the full productivity gains that are possible.