DeepSeek has released a new paper,Watch Mirror Twins Online with co-founder Liang Wenfeng credited as a contributor, detailing how its latest large language model DeepSeek-V3 achieves efficient training and inference using only 2,048 H800 GPUs – significantly fewer than the tens of thousands typically required. The team attributes this efficiency to four key innovations: memory optimization through multi-head latent attention (MLA), computational savings via a Mixture-of-Experts (MoE) design with FP8 precision, communication improvements using a multi-plane network topology, and faster inference through multi-token prediction (MTP). With MLA, KV cache memory usage is cut to just 70KB per token, up to 1/7 that of competing models. MoE architecture activates only 37 billion of the model’s 671 billion parameters per forward pass, reducing training costs by 90% compared to dense models. FP8 training further halves compute and memory usage, with minimal accuracy tradeoff. Beyond the model, the paper also outlines five future directions for AI hardware design, advocating for tighter integration between software and hardware to address memory, compute, and networking bottlenecks. [36Kr, in Chinese]
Related Articles
2025-06-26 11:33
924 views
NYT Strands hints, answers for May 5
If you're reading this, you're looking for a little help playing Strands, the New York Times' elevat
Read More
2025-06-26 11:23
1711 views
Announcing the Winners of 92Y's 2022 Discovery Contest by The Paris Review
Announcing the Winners of 92Y’s 2022 Discovery ContestBy The Paris ReviewMay 25, 2022ContestsT
Read More
2025-06-26 11:09
56 views
Postcards from Ellsworth by Rebecca Bengal
Postcards from EllsworthBy Rebecca BengalMay 19, 2022Arts & CultureEllsworth Kelly, Having The T
Read More