Terms and abbreviations

These entries locate explanations; they do not replace the conditions attached to each mechanism. Chapter numbers refer to the main text; A and B refer to the technical appendices.

Term	Meaning in this book	Where
Prefill / decode	Compute a prompt’s states / continue generating from cached states. Verification can evaluate several positions per request in a decode step.	1, 8
KV cache	Stored key/value information reused by attention. MLA changes its representation; it need not contain two separate arrays.	1, 3
TTFT	Time to first token, including waiting and work before publication.	2
$τ$ / SPS	Average useful output tokens per request per step / steps per second. Their product gives per-request token rate under the stated counting and measurement assumptions.	2, 8, 11
Pareto frontier	Non-dominated operating points: improving one objective requires worsening another among the compared points.	11
Roofline	A cost bound that compares arithmetic work with memory traffic; arithmetic intensity is operations per byte.	2
FLOP / MFU	Floating-point operation / model FLOPs utilisation. The chosen operation count and hardware denominator matter.	2
MLA	Multi-head Latent Attention: a shared compressed latent represents KV.	3
SWA	Sliding-window attention: a query sees a bounded recent interval.	3
RoPE	Rotary position embedding; a position-dependent transformation of attention queries and keys.	3
C1 / C2	One completed cache row per one / two token positions. Distinct from compression of a feature vector.	A
`ca_state` / `ca_cache`	Live score/latent compression state / completed compressed KV.	4, A
`ica_state` / `ica_cache`	Optional live index-compression state / completed index keys. The V4.1 KProject path needs no independent index state.	A
Top-k / top-p	Keep the highest-ranked k entries / a probability nucleus. Indexer top-k selects cache rows; sampling top-k selects possible tokens.	8, A
ZOS	Zero-overhead scheduling: overlap host work with the next forward while correcting information that is a step behind. The name does not guarantee that all host time is hidden.	1, 6
CUDA graph	Captured device work replayed using prepared buffers and a chosen size bucket.	6

Term	Meaning in this book	Where
HC	Hyper-connections: mix multiple residual streams around sublayers. Navigation only; the book does not derive this model mechanism.	B
Engram	The n-gram embedding mechanism; `EngramRuntime` fetches its sharded embeddings. A name, not an acronym expanded here. Navigation only.	B
MoE / EP	Mixture of experts / expert parallelism: route token work to expert weights distributed across ranks.	7
DP / CP	Data / context parallelism: engines handle different requests / workers cooperate within an engine. These groups can reuse the same ranks as EP.	7, B
EPLB	Expert parallelism load balancing: collect loads, choose replicas and placement, then move weights and update mappings.	7
MTP / nextn	Multi-token prediction / the configured draft count. A verification budget counts query positions, not just requests.	8
DFlash / DSpark	The draft-model path / the reported variable-length verification method. They are names; the implementation connects proposals, confidence and budgeting.	2, 8
YOCO / CED	You Only Cache Once / Causal Encoder-Decoder. Related model designs, not interchangeable names for every kind of cross-layer sharing.	9
Mask / compact	Two execution modes for truncated prefill: retain rows and mask work / physically remove rows. Their equivalence is distinct from agreement with full prefill.	9, 10
Batch invariance	The specified values remain identical when batch composition changes. State which values and comparator are tested.	10
Golden tokens	Prescribed token IDs that hold a test trajectory fixed while probabilities or other values are compared.	10
PIT / KS	Probability integral transform / Kolmogorov–Smirnov test. The sampler check uses randomised PIT values and tests their distribution.	10
PE / CE	Prefill engine / decode (completion) engine; they use the released `PROMPT` / `COMPLETION` roles. Separating roles does not supply production transfer and routing services.	5, 11
P/D; E/P/D	Prefill/decode; vision encoding/prefill/decode stage separation.	11
TTL / LRU	Time to live / least recently used. Retention time and capacity-driven eviction are separate mechanisms in the described service.	11
3FS	DeepSeek’s separately released distributed file system. The deployment account places block KV there, not SWA state.	11