Three Speedups That All Want the Same Thing
Speculative decoding, MoE serving and early exit, each evaluated against a fixed pool — why speculative decoding fits this hardware unusually well, why MoE is close to disqualified on one chip, and why all three compete with batching for the same surplus.