DeepSeek open-sources DeepInfer inference framework for DeepSeek-V4.1-Flash
Built on vLLM with custom kernels, the release contains offline inference only; the KV cache system and prefill/decode disaggregation needed for a full service are not included.
DeepSeek open-sourced DeepInfer on October 20, 2026 under the Apache 2.0 license. DeepInfer is the high-performance inference framework powering DeepSeek large-scale production services and currently supports DeepSeek-V4.1-Flash. It was built primarily on vLLM with inspiration from SGLang and TensorRT-LLM. Features include optimized kernels, attention data parallelism and balanced context parallelism, MoE expert parallelism with EPLB, DSpark with confidence-scheduled verification, decoder SWA bounded replay, asynchronous scheduling, batch-invariant inference, deterministic sampling and structured outputs. Supported hardware includes NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The release contains offline inference only; the KV cache system, prefill/decode disaggregation and data-parallel load balancing needed for a complete service are excluded.