Opening a series on serving an LLM to many tenants from one TPU chip — one HBM pool, one scheduler, a fixed $100 budget, and no horizontal escape hatch. The machine, the model, the objective, and what the constraint is for.
A deep-dive mental model for TPU system architecture — from single chips to Pod-scale distributed training, covering TPU VM, the MXU systolic array, ICI topology, XLA compilation, SPMD/GSPMD sharding, and the complete path from Python to silicon.
A ground-up explanation of three types of processors — what they are, how they work, and when to use each one. No prior hardware knowledge required.