#260 - Muse, Misalignment, Math

Andrey Kurenkov (0:11)

Hello, and welcome to the Last Week in AI podcast, where you can hear us chat about what's going on with AI.

As usual, in this episode, we will summarize and discuss some of last week's most interesting AI news. I'm one of your regular hosts, Andrey Kurenkov. I currently work at the AI startup Astrocade, and was at grad school for AI before that.

Jeremie Harris (0:31)

And hey, everybody, I'm your other co-host, Jeremie Harris from Cloudstone AI. I do AI and national security stuff, especially, you know, AI China Endgame. I actually have a newsletter now, so aiendgame.com. You can check that out if you want. We're probably, by the time this podcast drops, either we will have just released our latest post, sort of reverse engineering, with a couple people in the Frontier Labs, what recursive self-improvement looks like. Everybody talking about this, obviously, today.

We just wanted to make sure that someone somewhere had kind of, I guess, described at the Gears level what the process will look like inside the Frontier Labs, from the perspective of the researchers themselves who are doing it. If only for regulatory reasons, you need to be able to define something and understand it to regulate it. So this is kind of an initial attempt to do that. And just generally to trace out some of the espionage and national security implications of that Gears level picture. So if you want that, check it out. We already have a post up about US-China and kind of how we see the end game play out there. More of a diplomatic post, some of the things we've touched on on the podcast. A bit of a deeper dive and we'll probably do one at some point fairly soon, I think.

Andrey Kurenkov (1:34)

On that same topic. Yeah, we have a lot to dive deep on these days. Used to be, I'm a Sajik now for 2025, 2024

You know, it seemed like things are crazy back then, but things are really starting to get crazy. They're only going to get crazier by all appearances.

Jeremie Harris (1:52)

We don't have a story about this per se, but Jacob Coxson was on The Daily Show recently. I got a text from a buddy of mine passively following the space, probably following the space more than the average person. He just said, he wrote like, WTF, like, look at this, which I thought was so interesting because I thought that The Daily Show was so late on this one. I mean, the Coxson tweet, how long has it been? It's been like three weeks or something since the Coxson tweet. Obviously, there's been the hugging face thing it's been months and months ago. We've got a whole bunch of things. So I guess it's a way in which like, a lot of us are following, if you're listening to this podcast, you're probably not aware of how little the average person is following this stuff. Again, this buddy of mine is a really bright dude. He's in startups and biotech and following this stuff. And yet he still hadn't registered the Jacob Coxson story. So I just find it's fascinating, you know, like the future is here, it's not evenly distributed and often for funny reasons.

Andrey Kurenkov (2:44)

And the information of the future being here in some concerning ways and in some, I guess, exciting ways is also not equally distributed. And there's a lot of skeptics even who are sort of like doing counter narratives in any way.

But to focus on what we have today, it's actually gonna be a bit of a quieter episode. We've had quite a stretch of like really intense ones. And today there's not like a ton of exciting news. We'll have, you know, a bit of a mix of a couple tools, a couple stories, a couple business. Really the coolest news is about open source models rather unusually. For once, we'll get around to talking a bit more about research as well. So it should be a pleasantly unexciting episode in some sense.

We would like to thank Lankfuse for supporting the show. Lankfuse is the most widely adopted open source platform for AI, agent, evals and observability. In fact, it is a platform that my startup has been using for a very long time now, and I have personally grown to know quite a bit. It provides the tracing, evaluation, experimentation, and prompt management infrastructure that engineering teams need to build and continuously improve AI applications in production. By tracing, we mean that it captures everything with full execution of LLM-powered workflows, API calls, context, agent actions, costs, latencies, all these things that you would need as an engineer to debug when things go wrong or generally understand what your AI system is doing. In addition to that, you can set up evaluations, run experiments, and run systematic tests on what you've learned from production via LankFuse data sets. It is MIT licensed and self-hostable, and you have the option to use it via LankFuse Cloud as a managed service as well. And it should function with whatever stack you're using. It has over 100 integrations with things like OpenRouter, Versel, AISDK, LandGraph, and many more. It works with any model and any framework, so vendor lock-in is not a concern with LankFuse. Get started today with LankFuse Cloud at lankfuse.com. There's a generous free tier, no credit card required.