vLLM v0.28.0: the breaking change small GPU users must read

vLLM v0.28.0 shipped on 26 August 2026 with 584 commits from 270 contributors (76 new), and almost every headline in the release notes on GitHub is aimed at people running sixteen GPUs at once.

I read it from the opposite end: what happens to someone serving one model on one rented card. The answer is that two lines buried under "Breaking Changes" matter far more to that person than the entire Kimi-K3 section.

---

🧨 bitsandbytes is no longer in the box

The single most consequential line for a small-GPU setup: bitsandbytes support migrated to an out-of-tree plugin (#43529). bitsandbytes is how a lot of people load 4-bit and 8-bit weights onto a consumer card without pre-quantising anything. If your launch command passes --quantization bitsandbytes, a blind pip install -U vllm will not do what it did last week.

Here is the full breaking list, with who each one actually hits:

Change	PR	Who feels it
bitsandbytes moved out-of-tree	#43529	Anyone loading BnB 4-bit/8-bit weights on a small GPU
Transformers bumped to 5.15.0	#51668	Anyone pinning Transformers for another library in the same venv
calculate_kv_scales removed	#49389	FP8 KV cache users doing runtime scale calculation
override_attention_dtype removed	#48684	People forcing an attention dtype to dodge a numerics bug
reasoning_content output removed	#50624	Clients parsing reasoning traces from the response
KV tiering metrics renamed block → chunk	#52812	Anyone with a Grafana dashboard on offload metrics
MoE legacy code removed	#51078	Custom MoE kernels built against the old structure

Key takeaway: on a single-GPU box, v0.28.0 is not a "pull and restart" upgrade. Pin your current version, read those seven lines, and only then move.

---

📊 A doubled default that costs you memory

Under "New defaults", max_num_batched_tokens was raised from 8192 to 16384 (#51726). Prefix caching is now on by default for Mamba models (#50991), and the Blackwell CUDA graph capture default went up to 1024 (#49390).

These are throughput wins on big hardware. On a 16GB or 24GB card they are the opposite: a larger batched-token budget means more activation memory reserved before your KV cache gets a look-in. If you upgrade and hit an out-of-memory error you did not hit before, that default is the first thing I would check.

# Restore the old behaviour while you measure
vllm serve <model> --max-num-batched-tokens 8192

If you are sizing a card before you rent it, our LLM GPU Memory (VRAM) Calculator will tell you how much headroom you actually have to give away.

---

🖥️ The CPU and low-end story quietly got better

This is the part of the release I did not expect, and it is the part that matters most if your budget is a VPS rather than an H100 hour:

• An MLA backend for CPU, so DeepSeek-V2/V3 can run there at all (#49453)
• A triton-cpu wheel (#52092) and tcmalloc in the CPU path (#50841)
• GPTQ and AWQ enabled on s390x (#51148)
• An unquantized MoE backend for Power (VSX) (#51624)
• Disk offloading for the CPU offload connector (#49644), plus weight offloading in Model Runner V2 (#51413)
• An XPU wheel added to the release pipeline (#52108) for Intel cards

The CPU image is published like any other: docker pull vllm/vllm-openai-cpu:v0.28.0. None of this will be fast. But "slow and running on hardware I already pay for" beats "fast on a card I cannot afford" when you are learning, prototyping, or serving ten requests a day to a class project.

---

⚡ Speculative decoding is where the real speed went

Three of the release's headline items are speculative decoding: DFlash2 with local convolution and a candidate selector (#52816), DSpark confidence-scheduled verification (#47808), and async scheduling auto-enabled for draft models (#48341).

The eye-catching numbers in the notes belong to frontier setups, and I want to be precise about that rather than let them read as general claims:

Claimed number	What it actually applies to
~60% better TTFT	Adaptive speculative token budget, DSpark, on Kimi-K3 (#51725)
~17 GiB saved per GPU	Optional shared-expert sharding, Kimi-K3 only (#50912)
1.5~3x kernel-level speedup	Combined all-gathers, kernel level, not end-to-end (#51070)

None of those figures transfer to a 7B model on one card. Kernel-level speedup is not request-level speedup, and a Kimi-K3 memory saving means nothing if you were never going to load Kimi-K3.

Speculative decoding itself does transfer, and it is one of the few free speedups available on modest hardware. Before you wire up a draft model, run the numbers on our Speculative Decoding Speedup Calculator — the payoff depends almost entirely on the draft acceptance rate, and a bad draft model makes things slower, not faster.

---

🔐 The security note everyone exposing a port should read

Buried in the Security section is a documentation change worth more than most of the features:

vLLM's docs now warn that --api-key does not gate all endpoints (#51999).

If you are running vLLM on a cheap cloud box with a public IP and assuming that flag is your authentication layer, it is not. Put it behind a reverse proxy, or bind it to localhost and tunnel in. The same section also fixed a denial-of-service via sample-rate forgery that bypassed the audio decode duration guard (#49948), which is exactly the class of bug that bites a publicly reachable multimodal endpoint.

Two smaller hardening changes are the kind that break scripts quietly: cache_salt must now be non-empty (#50816), and non-object JSON bodies return 400 instead of 500 (#51654, #52528).

---

💡 What this means for you

If you are a student, freelancer, or small team in Sri Lanka running open models on rented or borrowed hardware, my read is:

1. Do not upgrade in place. Check the seven breaking changes against your launch command first. bitsandbytes is the likely one to catch you.
2. If you upgrade, pass --max-num-batched-tokens 8192 on the first run so you are comparing like with like, then raise it deliberately.
3. Ignore the Kimi-K3 and DeepSeek-V4 headlines. They are genuinely impressive engineering for clusters, and irrelevant to a single card.
4. Take the CPU and offload work seriously. Disk offloading and a CPU MLA backend widen what runs on hardware you can actually rent for a few dollars a month.
5. Fix your auth before your throughput. A public endpoint with no real gate is a bigger problem than 20% fewer tokens per second.

The wider pattern is worth naming. vLLM's release notes now read like infrastructure documentation for labs, but the project keeps shipping the unglamorous portability work in the same release. That second half is the part that decides whether people outside a handful of well-funded labs can serve open models at all. Before you commit to renting anything, price the options with our GPU Cloud Cost Calculator and check your expected throughput with the Inference Speed Calculator.