Moonshot AI open-sources Kimi K3 with 2.8T parameter MoE
Moonshot AI open-sourced Kimi K3 full weights on July 27 with 2.8T total / 104B activated parameters, 1M context, gaining 4000 likes in 30 minutes.
Moonshot AI open-sourced Kimi K3 full model weights on Hugging Face late on July 27, published the technical report, and simultaneously open-sourced three infrastructure technologies: MoonEP, FlashKDA, and AgentEnv.
Kimi K3 is a MoE architecture with 2.8T total parameters and 104B activated parameters, natively supports visual understanding, and features a 1M token context window. The model uses self-developed Kimi Delta Attention hybrid linear attention mechanism and Attention Residuals technology, activating only 16 out of 896 routed experts per token.
The open-source release uses the Kimi K3 License, an MIT-derived license. Hugging Face CEO Clem Delangue publicly stated that Kimi K3 received over 4000 likes within 30 minutes of launch, topping the platform's trending chart and setting the record for the fastest release growth in the platform's history.
vLLM and SGLang completed inference support on launch day, with vLLM reporting 118 tokens/s baseline and up to 370 tokens/s with speculative decoding on a 16x GB300 NVL72 cluster.