Open Source AI Is Getting Too Big to Run

**Ksenia** (0:00)
This week has shaken our understanding of how far an open-source model can go. Two models launched in the same week, one American and one Chinese, which often means bottle. Both are receiving praise from the open-source community, yet they could hardly be more different. One is deliberately not presented as the best. The other is beating Fable 5 and GBT 5.6 Sol in the Frontend Code Arena. Analyzed together, what do these two models tell us about where the industry is going?
Welcome to Attention Span by Turing Post. I'm Ksenia, and today we're going to examine these two strategies, why both models are so enormous, and what kind of control open weights provide when very few people can run them. The American model is Inkling, the first major model for MiraMurati's Thinking Machines Lab. MiraMurati used to be a CTO of OpenAI. The model has 975 billion parameters. It's a big model, and its weights are available to download right now. Thinking Machines says openly that the stronger models exist. The model is intentionally not the best out there, but the company is arguing that the ability to customize a model can count for as much as its initial benchmark position. The Chinese model is Kimi K3 from Moonshot AI. It has 2.8 Trugin parameters. Huge model. The previous Kimi model ranked 18th in the Frontend Code Arena. K3 entered at number 1 ahead of Claude Fable 5 So this is not a simple comparison between two models. Kimi is testing how close an open model can get the capability frontier. Thinking Machine is testing how much value can be created by adapting a model after it has been trained. A couple of words about being open. I used open source at the beginning because that is how these models are often discussed. For AI in general, open weight is the more accurate term. An open weight release gives you the model's learned parameters. You can host the model, compress it, fine-tune it and build a service around it. It does not necessarily include the trained data, the full training code or everything required to reproduce the model from the beginning. But using open weights allows you to fine-tune it and use it for your own needs. So, Inkling is available now. And Thinking Machines released the weights under Apache 2 together with an acceptable use policy. KimiK3's API is available. As I record this, its weights are not yet published. Moonshot says the full weights will be released by July 27th, together with more technical details. So, we have not yet seen the final license. Let's begin with Kimi, because its performance attracted the most attention. It blew up on Twitter, as traditionally happens on Twitter, now known as X. And when that happens, the servers often go down. So, if you are unable to try KimiK3 in the browser, wait a little. Something else will be released soon and everyone's attention will switch to that. Anyway, what happened and why is it so hot? KimiK3 shot up from 18th place to number 1 in the Frontend Code Arena. It scored 1679 points and ranked first in 6 of 7 categories, including consumer products, data and analytics, simulations, marketing and reference-based design. It came second in gaming behind Fable 5 The Frontend Code Arena uses blinded human comparisons. People look at two generated interfaces and choose the one they prefer without knowing which model produced it. This is where the claim that K3 is beating Fable 5 and GPT 5.6 Soul comes from. It is a strong result and it also is a specific result. It tells us that people preferred K3's Frontend Warp. It does not establish K3 as the best coding model for every task. On the broader artificial analysis intelligence index, K3 scores 57
Fable 5 scores 60, while the strongest GPT 5.6 Soul configurations score 58 or 59 Moonshot itself says that K3 still trails the strongest proprietary models overall. So, K3 is not winning everywhere and it's still absolutely extraordinary because a model scheduled for an open-weight release can now compete directly with the strongest closed models on visible useful work. Now, to the enormous number and how useful can this model can or cannot be for you. The model has 2.8 trillion parameters. K3 is a mixture of experts model. It contains 896 expert modules. These are groups of providers that develop different specializations during training. For each token, a router selects 16 of those 896 experts. 16 out of 896 Why?
Well, this reduces computation because K3 does not use all 2.8 trillion parameters for every token. Oh, that's why. It does not reduce the storage requirement in the same way, though. And all 896 experts still need to be available because the router can select a different group for the next token. What does this mean for you? If you have a regular Mac or one personal GPU, you cannot ram KimiK3 locally. Even compressed to 4 bits, the weights alone need roughly 1.4 TB of memory. A powerful consumer GPU has tens of gigabytes, not thousands. Moonshot recommends at least 64 accelerators for efficient deployment, so K3 becoming open-way will not turn into a local model. Most people will still use it through a website, an API, or a cloud provider. The practical benefit is that Moonshot may no longer be the only company allowed to host it. Other providers can run the same model, optimize it, compete on price, and offer different privacy or deployment options. For a regular user, that competition is the useful part of the open release. K3 is also not especially cheap by open model standards. Moonshot charges $3 per million on cash input tokens and $15 per million output tokens. Cash-out input costs $0.30 per million tokens. Another interesting moment is that price per token does not tell you the full cost. Artificial analysis reports that K3 generated 130 million output tokens across its intelligence index evaluation. The median model in its comparison group generated 63 million. Low reasoning can make a model more expensive even when the listed token price looks competitive. This is a change from the familiar open model proposition. K3 is not primarily offering a smaller and cheaper alternative. What it is offering is high capability with the possibility of independent hosts. Once the weights and license are available, Moonshot will no longer be the only organization able to operate K3. Another cloud provider can host it. A government can deploy it under its own operational rules. A company can choose a provider based on location, privacy requirements, performance, or price. Will American government use it? Doubtful, but wait for the next part. It actually might be the case also. So, most users will never operate 64 accelerators that Moonshot suggests for this model. They can still benefit if several companies are allowed to serve the same model. With a closed model, the developer usually controls the model, the API, the price and the hosting environment. Open weights allow these functions to be separated. One company can train the model and another can host it, and a third can adapt it for a customer. This is the control that Kimi may provide. It is control for operators first with indirect benefits for everyone who uses their service. Now to Inkling because there is an interesting part about Inkling and other services that Thinking Machines created. So let's look at Inkling. Inkling has 975 billion total parameters with 41 billion active for each token. It accepts text, images and audio. And a lot of people praise it for audio and supports a context window of up to 1 million tokens. On the Artificial Analysis Intelligence Index, Inkling scores 41 compared with K3 at 57 Thinking Machines says directly, Inkling is not the strongest overall model available today, open or closed. That is an unusual way to introduce a model after spending a great deal of money and time to train it. But Inkling makes more sense when you look at Tinker. Tinker is Thinking Machines' training platform. It allows customers to fine-tune models while Thinking Machines manages the distributed infrastructure. Tinker uses LoRa or low-reg adaptation. Instead of updating every provider in the base model, LoRa trades a much smaller set of additional weights. The base model stays largely unchanged while the adapt alerts a particular task, style, workflow or domain. This is much cheaper than retraining a giant model. It still requires good examples and reliable evaluation. If an organization cannot define the task or measure the result, fine-tuning will not solve the problem. When task is clear and the organization has useful data, a customized model can outperform a stronger general model on that specific work. In my conversations with other companies, I know that the companies that need precise work with their specific data, they use local models specifically for that. Here is the important business detail from Thinking Machines.