Ep. 033 - ClusterMAX 3.0 Is Here! Neoclouds Ranked (Neoclouds, GPUs) | Sam Harshe, Pratt Bhatt, Jordan Nanos

Jordan Nanos (0:16)

Hello everyone, welcome back to SemiAnalysis Weekly. We are here this week with a ClusterMAX 3 review podcast. I got Sam and Pratt, we're gonna talk through everything that was included in the article that we pushed out on Wednesday, September 23rd. We'll talk through the rankings as they are today, some of the statistics, like how many providers we actually tested, the methodology that we used to go through the testing over the last number of months, and tease some upcoming projects such as EndpointX, Post-TrainingX, HarnessX, SandboxX, we're gonna test everything.

And then talk about some of the trends that are affecting the Neocloud industry, financing, the move to Vera Rubin from Blackwell, Security, we had a previous article on that, Agenic Coding, how that's impacting infrastructure. And maybe we'll get into some individual stories about specific topics. But guys, welcome to the show. How is the last three months, Sam, you're coming out of your hole, man.

Sam Harshe (1:13)

Yeah, it's my first time seeing sunlight today, actually, since testing began. But yeah, I mean, I'm a little sunburnt from it, but I've had a few lifetimes worth of reps on Kubernetes and SERP. So I wouldn't trade for anything.

Jordan Nanos (1:29)

Pratt, how about you? You feeling good after the testing period has come to a close?

Pratt Bhatt (1:35)

Yep, feeling pretty good.

It was a lot of fun testing out these clusters. You know, always fun playing with big Infra.

Jordan Nanos (1:44)

Awesome. So headline news, I think the biggest thing is that there's a new addition to the Platinum tier. We've got Nebius up there joining CoreWeave. In the Gold tier, rounding it out, we've got Oracle and Google Cloud, Google joining Oracle. On the Silver side, Azure moves down, Firmus and Lambda stay, GMI moves up, and TensorWave stays at Silver.

And then in the Bronze, companies like AWS, Crusoe together all moving down, rounding out the list at a high level. This time we rank 77 providers. We increase our market view to cover 323 total, which is available for subscribers to our Neocloud model in great detail. And we've now interviewed well over 200 end users of Neoclouds as part of this research. So let me pose a question to Sam to start.

Most interesting insight that comes to mind when you think about the labs that we've interviewed so far, like reliability, performance, ease of use, support. What do you think is top of mind for people that are buying GPU clusters right now?

Sam Harshe (2:57)

It's a good question. I mean, the first thing that comes to mind is just the ridiculous back-gradation of the price curve at the moment. So people are willing to pay just about anything for any chips that they can get in the next few weeks, especially. You can get a big discount if you're willing to wait even a few months.

And yeah, I mean, that means that just about everyone in the industry looks like a genius right now. If you have chips to sell, even if your Vibe-Coded Kubernetes is completely broken, even if storage barely works, if you've got GPUs, you're going to find people willing to pay a pretty good margin to take them off your hands.

Jordan Nanos (3:35)

Yeah, it's interesting. We've talked about that trend a few times previously, which is like the providers that weren't able to sell their stuff early. Typically, that's a signal that they're not as high quality. But now, they're actually closing these deals for significantly better margins, which is interesting.

Pratt, how about you? We're expanding a little bit beyond ClusterMAX in the future. Things like serverless inference endpoints, posted post-training infrastructure for RL, harnesses, sandbox, you've been digging in there. When you're talking to the providers themselves about the roadmap, you find it confusing that people can't get Slurm and Kubernetes right sometimes, but they're already ready to launch like four new products.

Pratt Bhatt (4:18)

I mean, I guess, right? Because currently the margins on compute are so like high. Of course, like as a Neocloud, you do want to capitalize on that. You can, of course, like SpaceX provides really good compute to Google. They have deals with Anthropic and also several other Neolabs. However, Neoclouds also wants some of that cake. So they're thinking, what is the most high margin business that we can enter right now?

Managed clusters, of course, will be one of them, but it's a huge headache setting up the entire cluster. So why don't we just go above that layer and sell, actually, hosted training? Why don't we sell managed inference, you know? So they're just, in the end, just expanding their market, trying to earn the little good cash, which the labs are earning right now, but everything points to centralization of compute.