The anchor post of the series — the full request lifecycle from ingress to streamed token, which stages run on the host CPU versus the TPU, why the host starves the accelerator more often than anyone expects, and where admission control has to sit.
Opening a series on serving an LLM to many tenants from one TPU chip — one HBM pool, one scheduler, a fixed $100 budget, and no horizontal escape hatch. The machine, the model, the objective, and what the constraint is for.
Why the KV cache dominates memory on one chip, how PagedAttention makes the pool accountable, why cross-tenant prefix caching is a security question and not just a performance one, and the design space for budgeting a shared pool.
Every falsifiable claim the series has made, collected into one scoreable page — what I expect, how confident I am, and exactly what would prove each one wrong.
Precision as a capacity lever on a single chip — why bf16 and not fp16 on TPU, what FP8 calibration actually involves, and why weight quantization and KV cache quantization are two independent levers that multiply.
Runtime behavior on one node — why a cold engine takes minutes to become a serving machine, which knobs are operational and which are redeployments, choosing a degradation ladder before the machine chooses one for you, and why heterogeneous tenants are an asset.
Where the series stops describing hardware and starts writing policy — continuous batching, what fairness even means on one queue, priority tiers that do not starve, and how to pick an eviction victim without punishing your best tenant.