models / / #25
▲ MADE POSITIVE PROGRESS
PP8 current-state sweep + conflict flagged
PP8 (TP1xPP8) on our box: single-stream 43.2/40.9/32.2/27.6 @8/512/4k/32k; aggregate 4k c8 154.9, 32k c16 166.6, 128k c16 139.0, 512k c16 86.8; prefill 4k c1 1350 / c16 7067, 128k c16 30058. Conflict resolved by measurement: PP8 is ~2.3x SLOWER at c1 than TP8xPP1 (43 vs 100.8) - c1 -> TP8xPP1…
Benchmarks
| Metric | Value | Δ vs previous | Unit | Context | Note |
|---|---|---|---|---|---|
agg_decode_4k_tok_s (agg_decode_4k_tok_s) |
154.9 tok/s | -48.8% | tok/s | PP8 c8 | |
agg_decode_512k_tok_s (agg_decode_512k_tok_s) |
86.8 tok/s | -33.0% | tok/s | PP8 c16 | |
c1_decode_tok_s (c1_decode_tok_s) |
43.2 tok/s | -57.2% | tok/s | PP8, 8-tok prompt | |
prefill_tok_s (prefill_tok_s) |
30058 tok/s | 436.8% | tok/s | PP8 c16 @128k |
raw JSON: /api/reports/25