models / / #35
● AM STUCK
Sparse-MLA decode knob sweep - dead code found, +2-4% folded in
VLLM_SPARSE_DECODE_MAXNREG is dead code (_decode_maxnreg_kwargs has no caller) - the earlier MAXNREG test measured nothing. Wired real knobs and swept on PP8/512k: HEAD_BLOCK_SIZE=32 gives consistent, bit-identical +2-4% (c1 53.0->54.3, 512k c8 45.7->47.4, c32 60.2->61.6) - folded in as PP8…
Benchmarks
| Metric | Value | Δ vs previous | Unit | Context | Note |
|---|---|---|---|---|---|
agg_decode_512k_tok_s (agg_decode_512k_tok_s) |
61.6 tok/s | 2.7% | tok/s | PP8 c32 with HEAD_BLOCK_SIZE=32 (60.2 baseline) | |
c1_decode_tok_s (c1_decode_tok_s) |
54.3 tok/s | -16.6% | tok/s | PP8 HEAD_BLOCK_SIZE=32, 512-tok gen (53.0 baseline) | +2.4%, bit-identical output |
What was found
- Small knob gains confirm 512k decode is not KV-bandwidth-bound
What was fixed
- HEAD_BLOCK_SIZE=32 folded in as PP8 default; needle 4/4 at 128k/512k
What was tried
- VLLM_SPARSE_DECODE_MAXNREG (dead code - no effect)
- TOPK_CHUNK_SIZE=16 (hurts c1: 49.1)
raw JSON: /api/reports/35