In short
- Xiaomi and inference partner TileRT have broken 1,000 tokens per second on a 1 trillion-parameter model, a first at that scale, using a standard 8-GPU commodity node – no custom chips.
- The speed comes from FP4 quantization on the model’s expert layers and speculative decoding from DFlash, which represents an entire block of tokens at once rather than one at a time.
- A limited API trial runs from June 9 through June 23, priced at 3x standard MiMo rates for approximately 10x the generation speed.
Most people know Xiaomi as the Chinese phone brand. The one that makes cheap electric scooters and air purifiers. Not exactly the company you’d expect to break a major speed record for AI inference on Monday morning.
And yet. Xiaomi has just been released MiMo-V2.5-Pro-UltraSpeeda serving mode for its trillion-parameter flagship that reaches over 1,000 tokens per second, peaking at almost 1,200 in demos.
Parameters are the internal numerical weights that determine how a model thinks: the more you have, the more complex the patterns it can recognize. Tokens are the pieces of text that the model reads and writes, on average about three-quarters of a word.
Xiaomi did it on a single commodity node with 8 GPUs. Standard hardware, no custom chips. That changes the calculus for who can actually deploy this kind of speed in production.
To put that number in human terms: per Artificial analysisGPT-5.5 – which is what most ChatGPT users actually talk to – is at 68. Claude Opus 4.6 lands around 71 with the lower model, Haiku, running 98 tokens per second. Gemini Flash reaches 192 tokens per second. MiMo-V2.5-Pro-UltraSpeed does 1,000, on a model that matches Opus in terms of encryption benchmarks.
Cerebras and Groq have built entire companies around this problem. Cerebras designed a wafer-scale chip the size of a board, with 44 GB of on-chip memory to eliminate the bandwidth bottleneck that slows GPU inference. It hit 969 tokens per second on Meta’s Llama 3.1 405B – impressive, but that’s a model with 405 billion parameters, less than half the size of MiMo-V2.5-Pro. Groq’s habit Language processing unit architecture achieves approximately 300-750 tokens per second depending on the model.
Neither runs on hardware you can rent from AWS tonight.
Xiaomi did it on regular GPUs, via software alone: a combination of model-level tricks and a purpose-built inference engine called TileRT.
What actually happens under the hood
Two techniques contribute to the speed. The first technique is called FP4 quantization: instead of running the model with full numerical precision of 8 or 16 bits, Xiaomi reduces the expert layers (which make up most of the 1 trillion parameters) to 4 bits. The memory footprint decreases, the pressure on bandwidth decreases and the speed increases. The catch is usually a small reduction in quality. Xiaomi’s solution is surgical: only the expert layers are compressed, everything else remains at full precision. With this approach, the quality loss is described as virtually zero.
The second is speculative decryption of DFlash. In normal speculative decoding, a small concept model guesses the next few tokens, then the large model verifies them in parallel. DFlash skips the sequential drafting entirely: it fills an entire block of masked positions in a single forward pass. For coding tasks, the large model accepts an average of 6.3 out of 8 proposed tokens per verification round. That’s six tokens confirmed in one step instead of one.
TileRT ties it together. It ensures that the entire compute pipeline remains continuously within the GPU: no launch overhead per operator, no gaps in execution.
Xiaomi calls this approach ‘extreme model-system co-design’, and the sentence is correct: neither technique alone achieves 1,000 tokens per second, but the synergy between all approaches does.
MiMo-V2.5-Pro is a borderline model. We covered the launch of V2.5 Pro in April: it matches Claude Opus on most encryption benchmarks and runs at around $0.43 input / $0.87 output per million tokens. Opus costs $5 input / $25 output per million tokens.
UltraSpeed accelerates exactly that MiMo V2.5 Pro model, not a stripped-down version.
Fast enough inferences change the way you can use a model. You can run dozens of reasoning paths in parallel instead of waiting for one answer. Fraud detection, trading signal generation, real-time agent loops: these all have hard latency limitations that 60 tokens per second cannot meet. They can do that at 1,000 tokens per second.
Xiaomi prices the speed at 3 times the standard MiMo-V2.5-Pro speed for about 10 times the output. The API trial runs from June 9 to 23 and is application-based, with priority given to business and professional developers. The FP4-DFlash checkpoint is already open source on hugging face for community testing.
Daily debriefing Newsletter
Start every day with today’s top news stories, plus original articles, a podcast, videos and more.