DeepInfer: DeepSeek Open-Sources the Engine Behind Its Price List
On 2026-10-20 DeepSeek released DeepInfer, the inference framework behind its production services, under the Apache 2.0 license. It currently serves one model — DeepSeek V4.1 Flash 1017 — on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. It is not a turnkey API server: the KV cache system, prefill/decode disaggregation and cross-replica load balancing are not in this release. What it does publish is the single-deployment engine that lets DeepSeek sell DeepSeek V4.1 Flash at its current token price, which changes what every other host of the same weights can be expected to charge.
Batch-invariant inference and deterministic sampling (NVIDIA only)
Structured output
Regex, JSON schema and EBNF grammars
Published benchmarks
Pareto charts only: prefill on 4 SM100 GPUs, decode on 64; no numeric throughput in text
What was not released
Listed as out of scope in the README
Component
Why it matters
KV cache system
Prompt-cache hits are what make first-party cache pricing possible; operators must bring their own.
Prefill/decode disaggregation
The README says most engine functionality is implemented, but the orchestration is not released.
Cross-replica load balancing
Routing prefill and decode traffic across data-parallel replicas is left to the operator.
What it really means
It is an engine, not a service. DeepInfer covers what happens inside one model deployment — kernels, parallelism, speculative decoding, scheduling. The parts that turn a fleet into an API (cache tiering, disaggregated prefill/decode, traffic routing) stay private. Self-hosters get DeepSeek's per-GPU efficiency, not DeepSeek's fleet economics.
It is a fork of the open ecosystem, given back. The README says DeepInfer was originally built on vLLM. Expect the useful parts — EPLB expert balancing, DSpark verification scheduling, batch invariance — to be studied and ported into vLLM and SGLang rather than DeepInfer replacing them, at least while it supports a single model.
It reads as a reference implementation for V4.1 Flash. With one model supported, the immediate audience is anyone hosting those exact weights. The Decoder reports DeepSeek also released deepseek-recipe, which reproduces the official API's token processing, and says code for future models will follow.
Ascend support is a first-class target, not a port. Shipping the same production engine for Huawei Ascend 950 alongside NVIDIA SM100 is a public statement that DeepSeek's serving stack does not depend on a single chip vendor.
How it affects the inference industry
Before the release, Artificial Analysis measured ten endpoints serving identical V4.1 Flash 1017 weights between 70 t/s (DeepInfra) and 697 t/s (Baseten FAST), with cost per AA task from $0.24 to $2.03 (see the 1017 brief). Much of that spread is engineering. An open, production-grade engine compresses it.
Third-party hosts
The floor rises. Slow endpoints lose their excuse; fast ones must differentiate on fleet-level work (caching, routing, uptime, SLAs) rather than kernels. Margin on open DeepSeek weights is likely to thin.
Closed-model vendors
The Decoder frames the release as added price pressure on Western providers. The engine does not change any closed price list, but it makes a cheap, capable open model cheaper to self-host and to resell.
Hardware vendors
Day-one parity on Ascend 950 lowers the switching cost away from NVIDIA for DeepSeek workloads. Older NVIDIA generations are not listed, so this release favours the newest silicon.
Serving frameworks
vLLM and SGLang gain a reference for DeepSeek-specific optimisations. DeepInfer itself competes only for teams running V4.1 Flash on SM100 or Ascend 950.
Likely effects — and what to watch
This section is analysis, not reported fact.
Narrower endpoint spread. If providers adopt it, the next AA provider sweep for V4.1 Flash should show slow endpoints catching up. That is the clearest measurable test of the release.
Cheaper third-party prices for DeepSeek weights. Lower serving cost tends to flow into reseller prices on aggregators such as OpenRouter; watch for cuts on the V4.1 Flash listings.
Reproducible outputs as a feature. Batch-invariant inference and deterministic sampling (NVIDIA only) make identical-input, identical-output evaluation practical for self-hosters — useful for agent regression tests.
The missing pieces decide the endgame. If DeepSeek later releases its KV cache and disaggregation layers, self-hosting could approach first-party economics. Until then, DeepSeek's own API — with its off-peak discount — remains the cheapest way most teams can use the model.