Records the July 12-13 lab findings on one 6x RTX 5090 machine:
- vLLM-Moet comparison and why per-rank expert residency dominates
- colibri full-resident placement: 2.30 -> 6.28-6.84 tok/s, with the
tested-and-rejected directions documented
- MTP speculation: broken int4 head identified (issue #8), int8 head
reaches 69-79% acceptance but MoE verify batches scale expert cost
with S, so speculation loses at every depth; revisit after GPU
grouped GEMM
- AVX-512 int4 kernel qualification: numerically better than the old
order, quality-neutral (ppl delta 0.24%), +4-7% on CPU-heavy routing
README links the record from the honest-numbers section.
ZacharyZcR