DisTrO and the Quest for Community-Trained AI Models

**Jeffrey Quesnelle** (0:00)
So suppose tomorrow, every company said, we can't release open source models anymore. I'm sorry, we just, like, it's just bad business. Where would we be in the open source AI space? What are the technical problems that would keep us from being able to replicate that on ourselves? And it turns out the real big problem is that when it comes to training models, the current paradigm requires that all of the GPUs that train the model, they all have to be like in the same room. The way the technology stack has grown and assumes that all of the little brains that are training the model can talk to each other very fast. And if you are a single entity, you can just put it into one data center, it's all good. But we and sort of the collective we of the open-source AR movement aren't one entity. So how can we cooperate to actually train a state-of-the-art AI that we all own? It turns out that most assumptions in the AI space right now are a product of that's just how things had been done. So someone made an assumption maybe in the early 90s that everyone just kind of has continued to go with and we now can revisit it and look at those and see that there's actually a lot of growth there.

**Derek** (1:08)
Once again, thank you for listening to the a16z AI podcast. I'm Derek. You're about to hear a very interesting discussion between a16z General partner, Anjney Midha and Bowen Peng and Jeffrey Quesnelle from News. If you want to Google it, that's N-O-U-S Nous Research. They discuss it in more detail during the interview, but if you're not familiar with News, the short version is that it's a small team of researchers dedicated to making cutting-edge AI more accessible via open-source projects. Their guardrail-free Hermes models, for example, are quite popular in the AI builder community. However, the catalyst for this discussion is a recent paper the team released about a project called DisTrO. And DisTrO is an algorithm for training AI models on distributed infrastructure, utilizing the public internet. 100 megs down, 10 megs up. The News team claims DisTrO required 857 times less bandwidth than a standard approach to distributed training and could perform even better with optimal parameter tuning. Of course, anybody who follows AI closely knows that the kind of setup typically required to train even a reasonably sized model can be cost and skill prohibitive for all but the largest shops. And a big part of that is achieving the fastest possible communication among the GPUs in a cluster, which is why networking plays such a big role in the architecture of AI systems. Although DisTrO is very early research, the promise should be clear. Allowing small teams to train models using peer-to-peer networks, allow what projects like SETI at home or Folding at home did in their respective fields, could help break open AI innovation for the 99% of builders who don't have access to massive compute resources or don't want to be fenced in by the limitations of large AI labs. So, with that background, here are Anjney, Jeff and Bowen discussing Nous, DisTrO and more. It kicks off with Jeff explaining what Nous is trying to do and the unorthodox origin story of the team and how they met on Reddit.
As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a16z fund. For more details, please see a16z.com/disclosures.

**Jeffrey Quesnelle** (3:25)
Nous Research is an open source AI accelerator effort. So our goal is to accelerate artificial intelligence, and to do it in a way that brings it to everybody. When we say bringing it to everybody, it doesn't just mean that everybody can use it in the sense that it's a product that we bring, but it's that everyone can also touch the technology underneath it. So if you just want to use the AI, you can just use the AI. If you want to open up the code and see the code and use it, we believe that's as important as bringing the access of the actual using it, is also bringing the underlying technology to everyone. We've seen that open source innovation has driven, it's a multiplier in the technology stack. Every kid who wants to learn to do something starts by going with the open source free thing that they can get on the Internet. When they're 10, 12, what can I do and touch this? So our goal is to make sure that the building blocks of this transformative technology are kept open and that we're able to do the research to bring state-of-the-art AI to everyone.