Serving large language models in production often suffers from high per‑token latency and inefficient GPU utilization. Traditional autoregressive drafting improves acceptance but scales cost with block size, while fully parallel drafts are cheap but suffer rapid acceptance decay, wasting verification cycles on tokens that will be rejected. Teams need a method that keeps drafting inexpensive, preserves high acceptance deep into the block, and dynamically adjusts verification work to match current load so that throughput stays high without sacrificing quality.
DSpark addresses exactly these pain points. It pairs a cheap parallel backbone (like DFlash) that generates base logits for every position in a draft block with a lightweight sequential head that adds a prefix‑dependent bias before sampling each token. This hybrid approach inherits the parallel draft’s low cost while the sequential head steadies acceptance across the suffix, cutting the usual decay. A confidence head estimates the survival probability of each draft token, calibrated with sequential temperature scaling to stay accurate. A hardware‑aware prefix scheduler then decides how many tokens to verify per request: it verifies more when GPUs are idle and fewer when the system is busy, using an early‑stopping rule that guarantees lossless output. Offline tests show accepted length gains of 26‑31% over Eagle3 and 16‑18% over DFlash. In live DeepSeek‑V4 traffic, per‑user generation runs 60‑85% faster than the prior single‑token baseline, with no quality loss. The framework reuses existing model weights, adds only minimal overhead, and provides open‑source checkpoints and training code for easy adoption.
To get started, install the dependencies, run the provided training script against your target model, and evaluate with the evaluation script. Default configurations assume an 8‑GPU node but can be scaled down by adjusting visible devices. No retraining of the large model is required—just attach the draft module and deploy.
#AI #ML #LLM #Inference #Optimization #DeepSeek