The KV Cache Is the Whole Problem
Why the KV cache dominates memory on one chip, how PagedAttention makes the pool accountable, why cross-tenant prefix caching is a security question and not just a performance one, and the design space for budgeting a shared pool.