Partitioning by Hash of Key
Hashing the key to spread skewed data evenly — what a partitioning hash actually needs to be, why "consistent hashing" is the wrong name for it, and the range queries you give up to get the even distribution.
Systems
The Hot Key Hashing Can't Fix
Why a single very hot key defeats every partitioning scheme in this series, the random-suffix trick that splits it, and the read fan-out and bookkeeping you take on in exchange.
Why Partition at All
Why replication alone runs out of road, what a partition actually is under six different names, and why choosing how to partition is independent of choosing how to replicate.
Partitioning by Key Range
Assigning each partition a continuous span of the key space — why the boundaries have to follow the data, what it buys on range scans, and the hot spot a timestamp key walks straight into.
The Anatomy of a Request on One TPU Machine
The anchor post of the series — the full request lifecycle from ingress to streamed token, which stages run on the host CPU versus the TPU, why the host starves the accelerator more often than anyone expects, and where admission control has to sit.
What the Chip Is Actually Doing During Decode
Why decode is an awkward fit for a systolic array specifically, what Pallas is for, the three places idle time comes from on one chip, and an honest prediction about tensor parallelism that the hardware may well refute.
Toward Deploying a Multi-Tenant System on a Single TPU
Opening a series on serving an LLM to many tenants from one TPU chip — one HBM pool, one scheduler, a fixed $100 budget, and no horizontal escape hatch. The machine, the model, the objective, and what the constraint is for.
The KV Cache Is the Whole Problem
Why the KV cache dominates memory on one chip, how PagedAttention makes the pool accountable, why cross-tenant prefix caching is a security question and not just a performance one, and the design space for budgeting a shared pool.
Metrics, and How to Measure Them Without Lying to Yourself
The scoreboard for the rest of the series — the two latencies, throughput and what it hides, why every number is a per-tenant percentile, and HBM headroom as the one metric that only binds on a single machine.
Three Speedups That All Want the Same Thing
Speculative decoding, MoE serving and early exit, each evaluated against a fixed pool — why speculative decoding fits this hardware unusually well, why MoE is close to disqualified on one chip, and why all three compete with batching for the same surplus.
Predictions, Before the Hardware Can Argue Back
Every falsifiable claim the series has made, collected into one scoreable page — what I expect, how confident I am, and exactly what would prove each one wrong.
Quantization Is How You Buy Tenants
Precision as a capacity lever on a single chip — why bf16 and not fp16 on TPU, what FP8 calibration actually involves, and why weight quantization and KV cache quantization are two independent levers that multiply.
The Machine in Motion
Runtime behavior on one node — why a cold engine takes minutes to become a serving machine, which knobs are operational and which are redeployments, choosing a degradation ladder before the machine chooses one for you, and why heterogeneous tenants are an asset.
Multi-Tenancy Is a Scheduling Problem
Where the series stops describing hardware and starts writing policy — continuous batching, what fairness even means on one queue, priority tiers that do not starve, and how to pick an eviction victim without punishing your best tenant.
What "Concurrent" Actually Means
Why concurrent writes survive even a strict quorum, why last-write-wins is only safe for keys you never rewrite, and why two operations minutes apart can still be concurrent.
The Limits of Quorum Consistency
Why w + r > n is a probability dial rather than a guarantee — the edge cases that return stale data anyway, the consistency guarantees quorums never provide, and why staleness is so hard to monitor.
Sloppy Quorums, Hinted Handoff, and Multiple Datacenters
What a leaderless database does when a client cannot reach its quorum at all — why the fix buys durability rather than consistency, and how the same model stretches across datacenters.
Replication: What We've Learned
Closing the Database Replication series — the four reasons to replicate, the three approaches side by side, and the one trade-off that turns out to be underneath all of them.
Version Vectors, Siblings, and Tombstones
How a database tells an overwrite from a conflict — the version-number algorithm behind siblings, why merging by union resurrects deleted items, and what a version vector adds once there is no single replica.
Leaderless Replication
How Dynamo-style databases let any node accept writes, tolerate node failures without failover, repair stale replicas through read repair and anti-entropy, and tune the quorum condition w + r > n.
Multi-Leader Replication: When One Leader Isn't Enough
Three scenarios where multi-leader replication makes sense — multi-datacenter deployments, offline-capable clients, and collaborative editing — and why write conflicts are the price you pay.
Multi-Leader Replication Topologies
How multi-leader systems wire their nodes together — circular, star, and all-to-all topologies — and why all-to-all introduces a causality problem that version vectors solve.
Write Conflicts in Multi-Leader Replication
Why write conflicts are unavoidable in multi-leader setups, the full spectrum of resolution strategies from last-write-wins to CRDTs, and what actually counts as a conflict.
Consistency Guarantees Under Lag
Two more subtle problems caused by replication lag — monotonic reads and consistent prefix reads — and the approaches that address all three consistency guarantees together.
What Is Database Replication?
An introduction to why databases replicate data across multiple nodes, and the fundamental trade-offs that follow from that decision.
Replication Lag: The Hidden Cost
What happens when asynchronous replication lets followers fall behind — and the first concrete problem it causes: reading your own writes.
Leader-Based Replication: How It Works
How single-leader replication works under the hood — the leader-follower model, the replication log, and the trade-off between synchronous and asynchronous replication.
Inside Replication Logs
How databases actually ship changes from leader to followers — the four approaches to replication logs, their trade-offs, and why the format you choose affects everything from rolling upgrades to change data capture.
Running a Replicated Cluster
The operational side of leader-based replication — how to add a new follower without downtime, and what actually happens when nodes fail.