Why NVIDIA Hardware Struggles on Large Model Decode | Jason Goodison artwork

Why NVIDIA Hardware Struggles on Large Model Decode | Jason Goodison

MTS

November 1, 2026

General Compute co-founder and CTO Jason Goodison breaks down their $100M Series A to build a dedicated inference cloud on Cerebras wafers.

0:00

Jason Goodison

There's two workloads and inference. One of them is called Prefill, where you process the prompt, and the second one is called Decode, where you actually generate the tokens one at a time. And so what we do is we use Nvidia for the Prefill part, which is what they call Compute Bound, and TLDR is that Nvidia does a great job on it. Decode is where generating one token at a time is where Nvidia really struggles on the very large models.

And so our partners like Cerebras, that's their bread and butter, is taking a big model, splitting it up over a bunch of wafers and generating up to 2,000 tokens per second.

0:33

SPEAKER_2

Hello everyone. Welcome back to MTS. We are joined today with Jason Goodison. He's the co-founder and CTO of General Compute, the company that just raised $100 million Series A and announced it today that they're building an inference cloud on chips designed for inference rather than training. So Jason, welcome to MTS.

0:52

Jason Goodison

Thanks for having me. Glad to be here.

0:54

SPEAKER_2

Glad to have you. Big congratulations on the Series A announcement. Quite exciting. So what's the big reason? Why now? What did you guys officially announce today?

1:04

Jason Goodison

Well, we announced today that we raised, like you said, $100 million for our Series A, co-led by Blackbird and Abstract. I'm actually wearing the Blackbird merch today that was sent to me by the team there. So what we're using that money to do basically is to fund a bunch of the deployments that we already have signed in Q1 and accelerate our growth beyond that too. So we're deploying several stamps of 5 to 10 megawatt clusters in Q1.

We're using the money to lock in the energy assets, which we've locked in many of them already, secure the chips, secure the capacity, networking, cooling, all of that kind of fun stuff. So we can do actual physical deployments and I think we're super excited about it.

1:49

SPEAKER_2

Congratulations. For people who aren't familiar with General Compute, one of the big bets here is that fast inference actually needs specialized chips. That's why you guys were able to close some of those early partners. Tell me a little bit more about this problem space. What are you guys working to solve?

2:04

Jason Goodison

Yeah, I think the right way to think about it is something like, not everybody drives the same car.

Some cars are better for some people than for others. And so in the same way that some cars work for some people and not others, some workloads require certain silicon that is optimized on different metrics. So the most obvious one is just latency, like ends to end generation time. We're finding that AI coding agents, we want the answer as soon as humanly possible.

And when we try to run very large models on Nvidia hardware primarily, we find that the profitability of the racks goes down to the bottom of the barrel. And it's really hard to make a ton of money and to actually get them to run and operate fast. But there's two workloads and inference. One of them is called prefill, where you process the prompt, and the second one is called decode, where you actually generate the tokens one at a time. And so what we do is we use Nvidia for the prefill part, which is very, what they call compute bound. And TLDR is that Nvidia does a great job on it. Decode is where, generating one token at a time is where Nvidia really struggles on the very large models. And so our partners like Cerebras, that's their bread and butter, is taking a big model, splitting it up over a bunch of wafers and generating up to 2,000 tokens per second. So the very kind of latency sensitive applications work exceptionally well on hardware like Cerebras. And we're the deployment partners. We're the ones that actually buy the hardware, deploy it and put it into customers' hands.

3:37

SPEAKER_2

Customers' hands, speaking of customers' hands, I know you guys kind of alluded to some of these customers and some of these contracts that already closed. Anything you can share there?

3:48

Jason Goodison

I can't share much, but we will have some announcements in the coming weeks. And we're very proud of the customers that we operate with.

3:56

SPEAKER_2

Okay, we don't have to get into any specifics, but at a high level, why do you think customers are signing these long contracts with you guys? What's the big reason here?

4:05

Jason Goodison

Well, there's two reasons. The first is that the hardware is having its moment, its product market fit moment. I think we're in the dial up phase of AI right now, where everything is extremely slow. And people think that it's normal to run a few agents at a time and to context switch all day and to go for a cup of coffee while something is running.

Read the other 12 minutes of this episode here.

$15 gets you this episode and 99 more. Credits never expire. After checkout, one link brings you back here to read it.

Not what you expected? Full refund within 14 days.

Thousands of transcripts fetched by people building searchable podcast archives

Fetch transcripts with code

The same credits work from a script or an AI agent. The demo key returns a sample episode in full, no card needed:

request
curl -H "x-api-key: pt_demo" \
  https://spoken.md/transcripts/1000651996090

Markdown with the speakers named, for your notes, your knowledge base, or anything that makes HTTP calls.

From $0.10 per transcript. No subscription. Credits never expire. Prices exclude VAT, added at checkout for EU customers. Not what you expected? Email us within 14 days with 20 or fewer credits used and we refund the pack in full.

Using your own key:

request
curl -H "x-api-key: YOUR_KEY" \
  https://spoken.md/transcripts/1000799237767