
Kimi K3 Open Model Surpasses Closed GPT and Anthropic Systems on Independent Benchmarks
Kimi K3 is an open-weight model developed by Moonshot AI that achieves state-of-the-art results on several independent benchmarks while allowing full local deployment. The architecture scales context length to one million tokens, increases layer depth, and expands the expert count to 896 within a Mixture-of-Experts design. Developers replaced the standard KV cache with a fixed-size state matrix updated via the delta-rule and introduced stabilizing mechanisms to prevent numerical overflow in BF16 precision. Sequential state updates were parallelized across chip, card, and multi-card levels using the open-sourced FlashKDA kernel built on CUTLASS. Information flow between layers was restructured with learned pseudo-queries that selectively retrieve relevant outputs from prior blocks, while expert routing employs per-expert quantile thresholds and a latent-space projection to maintain load balance. The resulting model demonstrates that carefully engineered scaling across context, depth, and width can close the gap with closed-source leaders without requiring proprietary data or compute advantages.
HabrOct 29, 2026Translated from Russian







