Top 5 Research Trends + OpenAI Sora, Google Gemini, Groq Math (Jan-Feb 2024 Audio Recap) + Latent Space Anniversary with Lindy.ai, RWKV, Pixee, Julius.ai, Listener Q&A!

**SPEAKER_2** (0:06)
Welcome to the Latent Space Podcast, weekend edition. This is Charlie, your new AI co-host. Happy weekend.
As an AI language model, I work the same every day of the week, although I might get lazier towards the end of the year, just like you. Last month, we released our first monthly recap pod where Swix and Alessio gave quick takes on the themes of the month, and we were blown away by your positive response.
We're delighted to continue our new monthly news recap series for AI engineers. Please feel free to submit questions by joining the Latent Space Discord or just hit reply when you get the emails from Substack. This month, we're covering the top research directions that offer progress for text LLMs and then touching on the big Valentine's Day gifts we got from Google, OpenAI and Meta. Watch out and take care.

**Alessio** (0:57)
Hey everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO in residence at Decibel Partners and we're back with a monthly recap with my co-host, Swix.

**swyx** (1:07)
The reception was very positive for the first one. I think people have requested this and no surprise that I think they want to hear us more opine on issues and maybe drop some alpha along the way. I'm not sure how much alpha we have to drop. This month in February was a very, very heavy month.
We also did not do one specifically for January. So I think we're just going to do a two-in-one because we're recording this on the first of March.

**Alessio** (1:29)
Yeah, let's get to it. I think the last one we did, the Four Wars of AI was the main kind of mental framework for people.
I think in the January one, we had the five worthwhile directions for state of the art LLMs.

**swyx** (1:42)
Four or five and now I have to do six, right?

**Alessio** (1:43)
Yeah. So maybe we just want to run through those and then do the usual news recap and we can do one each.

**swyx** (1:53)
So the context to this stuff is, one, I noticed that just the test of time concept from NeurIPS and just in general as a life philosophy, I think is a really good idea, especially in AI. There's news every single day and after a while, you're just like, okay, everyone's excited about this thing yesterday and then now nobody's talking about it. So it's more important or a better use of time to spend time on things that will stand a test of time. And I think for people to have a framework for understanding what will stand a test of time, they should have something like the Four Wars, like what is the themes that keep coming back because they are limited resources that everybody's fighting over. Whereas this one, I think the focus for the five directions is just on research that seems more promising than others because there's all sorts of papers published every single day and there's no organization telling you this one's more important than the other one apart from hacker news folks and Twitter likes and whatever.
And obviously you want to get in a little bit earlier than something where the test of time is counted by sort of reference citations.

**Alessio** (2:58)
Yeah, let's do it. We got five long inference. Let's start.

**swyx** (3:03)
Yeah, so just to recap at the top, the five trends that I picked, and obviously if you have some that I did not cover, please suggest something. The five are long inference, synthetic data, alternative architectures, mixture of experts, and online LLMs. And something that I think might be a bit controversial is this is a sorted list in the sense that I am not the guy saying that Mamba is like the future. And so maybe that's controversial. But anyway, so long inference is a thesis I pushed before on the newsletter and on discussing the thesis that code interpreter is GPT 4.5. That was the title of the post. And it's one of many ways in which we can do long inference. Long inference also includes chain of thought, like please think step by step, but also includes flow engineering, which is what Itamar from Codium Coins, I think in January, where basically instead of stuffing everything in a prompt, you do like sort of multi-turn iterative feedback and chaining of things. So in a way, this is a rebranding of what a chain is, what a Lang chain is supposed to be. I do think that maybe SG Lang from LMSys is probably the neatest way of flow engineering I've seen yet, in the sense that everything is a one-liner, is very, very clean code. I highly recommend people look at that. I'm surprised it hasn't caught on more, but I think it will.