Jonathan Ross, Founder of Groq

**David Senra** (0:02)
Let's start with this $20 billion, rumored $20 billion partnership that you have with Nvidia. Can you talk about the structure of the deal and how it came about?

**Jonathan Ross** (0:10)
So, the most interesting part about it is the call where the idea was first floated was about three weeks before money was in the bank.

**David Senra** (0:20)
That was a chance to move fast.

**Jonathan Ross** (0:21)
Of course, that's how you stay ahead.

**David Senra** (0:24)
So how did it come about?

**Jonathan Ross** (0:25)
So we had been working on integrating GPU and LPUs together. The best way to describe why this helps is if you're building out a logistics network for the United States, and I told you you could have either 18 wheelers or vans for last mile delivery, which one would you pick? And the answer is both, right? And so GPUs and LPUs combined ended up giving better performance across the performance curves.
We had implemented it and we had gone to Jensen asking, could we buy about 100,000 GPUs because we were going to deploy them ourselves.
And Jensen saw what we had done and thought, maybe it would be better to make this available to all of their customers.

**David Senra** (1:06)
You and I had this conversation at Nvidia GTC, and you were talking about the fact that these technologies are very complimentary. Can you explain a little bit more about that?

**Jonathan Ross** (1:14)
When you're processing an LLM token, what's happening is you're doing all of these different matrix multiplies. And some of them are more compute constrained, and some of them are more memory throughput constrained. And the ones that are more compute constrained, we put on the GPU, and the ones that are more memory throughput constrained, we put on the LPU.
And the bottlenecks are all over the place. There's all sorts of different bottlenecks. There is no one strategy. There is no one perfect architecture. So the realization was you put these two things together, and you defeat the bottlenecks across all of the different matmols.

**David Senra** (1:46)
There's another thing that I love that you said when we had this conversation that when AI is talking to other AI, speed is becoming more and more important.

**Jonathan Ross** (1:53)
Yeah, I mean, a human can wait a second or two to get a response when they type a command into a computer. AI is just sitting there waiting because it produces these tokens so much faster, it thinks so much faster. Now, you bring in LPUs and the speed is so much faster that it just becomes all about how do you move as fast as you can. AI is really good at using AI.
That's really what agentic is, right? So humans benefit from using AI, so does AI.
Just like you would do research about me before I show up on your show, AI is going to kick off a job doing research on different tools it's going to use while it's using this other tool. And so it kicks it off to another AI. And so you get this exponential growth.

**David Senra** (2:35)
I'm going to go on a tangent just for a second, and then we'll come back to this. I talked to a lot of founders about this recently, but the whole point is like when agents are making payments, the amount of payments that are going to be made is going to skyrocket. Do you have any insight into that?

**Jonathan Ross** (2:47)
Yeah, I think it's still early, and one of the limiters is payments aren't really built for this yet. But if you can make micropayments, the number of payments is going to skyrocket. I did a little hobby project, and for the hobby project, I needed a couple of different phone numbers, so I could have a bunch of different agents on with me on Signal and WhatsApp.
I had to go to Twilio, and I had to prove I was a human in order to get the number and all this stuff, and it was just this big pain to get it done. If on the other hand, if I had allocated a budget to the AI and could have just used that budget, they would have spent it, and I would have never known, it would have just been within the budget.

**David Senra** (3:30)
Tell me more about the hobbies and the side projects, because this has come up in a couple of our previous conversations that you and I have had.

**Jonathan Ross** (3:36)
I like to do cutting edge stuff on my personal computer, so I don't have access to the work code base. Things where there's risk, I'll spin up a server in GCP or AWS, and I'll just start building stuff. I built everything from apps that tell you what airplanes to take if you're traveling, and what routes, and which routes have the best seats, and all this stuff, all the way to apps that do certain mathematical things, like a daily brief and all of this stuff. Very simple stuff. At work, I use it very extensively, but I always start with a hobby project before I bring it to work.