How DSpark Speeds Up LLM Inference by Deciding What Not to Verify

DeepSeek and Peking University’s open speculative-decoding stack: 60–85% faster per user on V4, with drafters you can train for your own models

Jun 30, 2026

In ~7 mins: the two moves behind DSpark, the offline gains over DFlash and Eagle3, the 60–85% production result, the one reproducibility catch, and a runnable train-to-eval appendix.

DeepSeek just made its V4 models 60% to 85% faster per user, without retraining a single model weight.

The method is called DSpark, and it shipped open source under an MIT license.

Its real move is not a faster draft model. It is deciding which guesses are even worth checking.

Context

The research comes from DeepSeek-AI and Peking University, in a paper titled “DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation.” It landed on June 27, 2026, alongside DeepSpec, an MIT-licensed repo for training and evaluating draft models.

Most people saw it through a post by Daniel Han.

A quick refresher on what it speeds up. Speculative decoding is the standard way to make generation faster without changing the output.

A small draft model guesses several tokens ahead, the full model checks the whole guess in one pass, and it keeps the longest run that matches. Because the check uses rejection sampling, the final text is identical to what the big model would have written alone.

There are two ways to build the draft model, and each has a flaw. Parallel drafters write the whole guess in one shot, which is fast, but the later words drift because each position is picked without seeing the ones chosen before it.

The textbook failure is blending two valid phrases like “of course” and “no problem” into “of problem.” Autoregressive drafters avoid that by writing one word at a time, but writing word by word is slow.

DSpark keeps the fast parallel draft and adds a tiny corrector on top that nudges each word based on the one just picked. Then it adds a second small model that scores how likely each guessed word is to pass the full model’s check.

A scheduler reads those scores and the current GPU load, and sends only the worthwhile guesses for verification. When the GPUs are busy, it stops checking guesses that probably will not survive.

Draft better, verify smarter

DSpark improves two of the three levers that set speculative-decoding speed. Per-token latency is the draft time plus the verify time, divided by how many tokens get accepted per round.

You can draft faster, accept more, or verify cheaper. DSpark does the last two.

Draft better: a parallel backbone with a sequential head.

The heavy work stays in a parallel backbone, the same DFlash design that produces a full block of guesses in one forward pass. On top sits a lightweight Markov head that adjusts each word’s odds using only the word right before it.

It is stored as a small rank-256 factorization, so it adds almost no cost.

Adding it raises round latency by 0.2% to 1.3% at batch size 128, and in return the model accepts up to 30% more tokens. A two-layer DSpark beats a five-layer DFlash, so the correction earns more than extra depth would.

Figure 1: DSpark drafts a block with the parallel backbone plus sequential head, scores each token, keeps the high-confidence prefix, and verifies it in one target pass.

Figure 2 shows why the head matters. DFlash starts strong on the first token because it can afford a deep network, then decays on later positions.

Eagle3, the autoregressive baseline, starts lower but holds steadier. DSpark keeps the strong first token and flattens the decay, which is the behavior that lifts the whole block.

Verify smarter: confidence scores plus a hardware-aware scheduler.

A second small head scores each guessed token with the probability it survives verification, assuming everything before it was accepted. Its training target is the exact per-step acceptance rate, derived from how far the draft’s distribution sits from the model’s own.

Those raw scores run overconfident, so DSpark recalibrates them with a step called Sequential Temperature Scaling. Calibration error drops from a range of 3% to 8% down to about 1%, which matters because the scheduler needs real probabilities, not just a ranking.

The scheduler then treats verification length as an allocation problem. It multiplies expected accepted tokens by the engine’s measured throughput at a given batch size, sorts every candidate guess by its survival odds, and admits them one at a time while total throughput keeps rising.

It stops the moment throughput would fall. That stopping rule is also what keeps the method lossless.

A counterexample in the paper’s appendix shows a greedier search would quietly shift the output distribution, in their example from (0.7, 0.3) to (0.85, 0.15).

The payoff is sharpest on open chat, where pruning low-confidence guesses raises the acceptance rate from 45.7% to 95.7%. Structured math and code start higher and move less.

What it scores on the benchmarks.

Offline, with the scheduler turned off to isolate raw draft quality, DSpark raises accepted length over Eagle3 by 27% to 31% and over DFlash by 16% to 18% across Qwen3-4B, 8B, and 14B. The gains hold on Gemma4-12B, which is the harder test, since speculative decoding often fails to transfer across model families.

Production is where the viral number comes from. Deployed inside DeepSeek-V4 against the previous MTP-1 baseline, DSpark runs 60% to 85% faster per user on V4-Flash and 57% to 78% faster on V4-Pro at matched throughput.

Aggregate throughput rises about 51% to 52% at moderate latency targets.

The 661% and 406% figures circulating on X are real but narrow. At very strict latency targets the old MTP-1 baseline nearly falls over and can serve only a tiny batch, so the ratio balloons.

DeepSeek’s own paper reads those points as proof that DSpark can serve an interactivity tier MTP-1 could not, not as a six-times speedup anyone should plan around.

Under load, the scheduler expands verification from MTP-1’s fixed 2 tokens to roughly 4 to 6 per request when there is room, then pulls back as concurrency climbs.

How to run it

Here is the part that decides whether this helps you. The open DeepSpec repo lets you train and evaluate a DSpark draft model for your own Qwen3 or Gemma target and reproduce the offline numbers above.

That path works today.

The full production speedup is a different story. It rides on serving-engine changes DeepSeek made to V4, including variable-length verification and two custom attention kernels, and those are described in the paper rather than shipped in the repo.

vLLM support is still in open pull requests as of late June. Full install-to-evaluate commands are in the appendix below.

AlphaSignal Take

DSpark is a real result, not a benchmark trick. The offline gains are measured with the scheduler off, the production gains come from live traffic, and the whole training stack is open under MIT.

The cross-family transfer to Gemma is the part we did not expect to hold.

The reproducibility gap is the catch. The open checkpoints and configs reproduce the Qwen3 and Gemma offline results, not the V4 production speedup.

V4 uses a different draft config and serving kernels that are not in the repo, so the 60% to 85% number is not something you can clone and rerun.

The draft cost is unrecoverable on hard prompts. This is the paper’s own caveat.

DSpark always pays to generate the full draft block first, and for low-acceptance queries that compute is wasted before the scheduler can prune anything.

Adoption is not free. The default data-prep step for a Qwen3-4B drafter warns of a roughly 38 TB target cache, the configs assume a single 8-GPU node, and the README says to re-finetune the drafter for thinking-mode targets.

What the announcement glosses over is that the public configs are not the V4 production configs, so the most-shared number is the least reproducible one. The piece worth watching is whether vLLM lands native DSpark support, because that is what turns this from a DeepSeek-only win into something the rest of us can serve.

So the best recommendation is to adopt it now if you train your own drafters for Qwen3 or Gemma, and to wait if you want turnkey V4-level gains in your own serving stack.

The method is sound. The production payoff is gated on engine support that is still landing.

Which lever is your bottleneck right now: draft quality, or how much you verify under load?

Appendix: How to actually run DSpark

This is a shorter path than the repo README.

Part A is runnable today. Part B is what production needs and the repo does not ship.

Part A: train and evaluate a DSpark drafter (runnable today)

Install dependencies (CUDA build of torch to match your machine):

pip install -r requirements.txt
# torch==2.9.1, transformers==5.10.2

2. Pick a config. DSpark configs live in config/dspark/: dspark_qwen3_4b.py, dspark_qwen3_8b.py, dspark_qwen3_14b.py, dspark_gemma4_12b.py. Defaults are block_size=7, num_draft_layers=5, markov_rank=256.

3. Prepare data. The pipeline downloads and splits prompts, regenerates answers with the target model, then builds a target cache. Budget storage first: the default Qwen/Qwen3-4B setting warns of roughly 38 TB for that cache.

bash scripts/data/prepare_data.sh

4. Train. One worker spawns per visible GPU, and checkpoints write to ~/checkpoints/<project_name>/<exp_name>/step_*. The default scripts assume one 8-GPU node.

bash scripts/train/train.sh
# point config_path at e.g. config/dspark/dspark_qwen3_4b.py

5. Evaluate on the speculative-decoding benchmarks (gsm8k, math500, aime25, humaneval, mbpp, livecodebench, mt-bench, alpaca, arena-hard-v2). Set target_name_or_path to the target the drafter was trained against and draft_name_or_path to your checkpoint. The --confidence-threshold flag controls offline pruning, where 0.0 verifies the full block and records calibration metrics.

bash scripts/eval/eval.sh

6. Skip training if you only want to reproduce Table 1. Pull a released checkpoint from the README table (Qwen3-4B/8B/14B and Gemma-4-12B-it, for Eagle3, DFlash, and DSpark), for example deepseek-ai/dspark_qwen3_4b_block7, and run step 5 against it.

Part B: what production needs (paper-described, not in the repo)

The V4 deployment is not a config swap. DSpark-5 there uses a max draft length of gamma=5, a parallel backbone of three MoE layers with mHC and sliding-window attention of 128, and a confidence head calibrated with Sequential Temperature Scaling.

The scheduler runs asynchronously to fit Zero-Overhead Scheduling and CUDA graph replay, approximating the next batch’s capacity from confidence outputs two steps earlier. Variable-length verification inside a batch needs tokens flattened across requests plus a marker tensor in sparse attention, and on V4 only the index-attention and compress kernels were modified.

None of that serving code ships in DeepSpec.

If you want this in a standard stack, track the vLLM work before planning anything: pull requests #46965, #46995, and #47093, and feature issue #46910.

AlphaSignal

Discussion about this post

Ready for more?