Kimi-K3.1-Pro-DSpark: An Open Draft Model for Faster Kimi K3.1 Pro

Today we are releasing Kimi-K3.1-Pro-DSpark, an open-weight draft model that speeds up decoding for Moonshot AI's Kimi-K3.1-Pro through speculative decoding. In each step the draft proposes a block of tokens and Kimi-K3.1-Pro verifies the whole block in one forward pass. Every token that is kept is drawn from the target's own distribution, so the outputs are exactly those of Kimi-K3.1-Pro; only the speed changes.

At block size 8, the draft commits 3.45 tokens per verify step on average across ten public benchmarks, instead of one token per step without speculation, and 3.28 on 95K-token prompts.

What is in the draft

The draft is a DSpark head: a small transformer that reads hidden states the target has already computed and drafts a whole block of tokens in one parallel pass.

• Backbone: 5 dense decoder layers, hidden size 7168, grouped-query attention with 64 query and 16 KV heads; 2.25B parameters in BF16.
• Target features: hidden states from target layers 7, 23, 67, 83 and 91 of 93, fused by one projection. The set reaches into the last layers of the target, where its states are closest to the next token.
• Block dependency: a low-rank Markov head (rank 256) conditions each drafted token on the one before it, so a block reads as a sequence rather than seven independent guesses.
• Confidence head: a per-position estimate of whether each drafted token will be accepted, which a scheduler can use to size verification.
• Shared with the target: the token embedding and the LM head, so the release adds only the draft layers.

Acceptance across ten public benchmarks

We ran each benchmark on its full public test set: GSM8K, MATH-500, AIME 2026, GPQA Diamond, HumanEval, MBPP, SWE-bench Pro, MT-Bench, the 11-category SPEED-Bench qualitative split and its 10K-token low-entropy throughput split, plus AA-LCR for long context. That is 5,399 requests, every one of which completed.

Acceptance is highest where the target's next tokens are most predictable: short math solutions reach 5.14 on GSM8K and 4.37 on MATH-500, and code stays near 3.5. It holds up at long context too: on AA-LCR's 71K to 115K-token document sets the draft still commits 3.28 tokens per step.

How we measured. Each prompt is one user turn in Kimi-K3.1-Pro's native chat template at maximum thinking effort, sampled at temperature 1.0 and top-p 0.95 on one 8x B300 node. For each request, acceptance length is completion tokens divided by verify steps; the numbers above average it over requests. Per-benchmark counters, acceptance histograms and per-position acceptance rates are in benchmark_results.json next to the weights.

Try it

With SGLang:

python -m sglang.launch_server \
  --model-path moonshotai/Kimi-K3.1-Pro \
  --trust-remote-code \
  --tp-size 8 \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path LithosAI/Kimi-K3.1-Pro-DSpark \
  --speculative-dspark-block-size 7 \
  --speculative-num-draft-tokens 8

With vLLM:

vllm serve moonshotai/Kimi-K3.1-Pro \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --speculative-config '{"method": "dspark", "model": "LithosAI/Kimi-K3.1-Pro-DSpark", "num_speculative_tokens": 7}'

The weights are released under the Kimi K3.1-Pro License of the target model. Faster decoding is the core of what we build at LithosAI, and we will keep releasing the pieces that help everyone run open models faster.