#architecture

September 7, 20267 min readTPU

The Anatomy of a Request on One TPU Machine

The anchor post of the series — the full request lifecycle from ingress to streamed token, which stages run on the host CPU versus the TPU, why the host starves the accelerator more often than anyone expects, and where admission control has to sit.

#tpu #llm #inference #vllm #architecture #multi-tenancy

July 15, 202642 min readTPU

Google Cloud TPU System Architecture: From First Principles to Pod-Scale Training

A deep-dive mental model for TPU system architecture — from single chips to Pod-scale distributed training, covering TPU VM, the MXU systolic array, ICI topology, XLA compilation, SPMD/GSPMD sharding, and the complete path from Python to silicon.

#tpu #google-cloud #machine-learning #distributed-training #architecture #xla #jax