Why decode is an awkward fit for a systolic array specifically, what Pallas is for, the three places idle time comes from on one chip, and an honest prediction about tensor parallelism that the hardware may well refute.
Speculative decoding, MoE serving and early exit, each evaluated against a fixed pool — why speculative decoding fits this hardware unusually well, why MoE is close to disqualified on one chip, and why all three compete with batching for the same surplus.