Lab

Processing a token requires a lot of intermediate calculations — normalizing activations, projecting into query/key/value space, computing attention scores, running through the FFN. Each of these produces numbers that are only needed for that one token's processing. Allocating and freeing fresh memory for each token would be slow and fragmented. Instead, malloc_run_state() allocates a set of reusable buffers once at startup. Every token reuses the same memory — the previous values are simply overwritten.

What each buffer does in plain English: - x holds the current token's representation as it flows through each layer (1024 numbers — the "residual stream") - xb, xb2, xb3 are temporary copies of that representation used during normalization and attention - q, k, v hold the query, key, and value projections for the current layer - att holds the attention scores — how much each past position contributes to the current one - logits is the final output — 151,936 scores, one per possible next token - hb and hb2 are FFN workspace — they hold the expanded intermediate representation during the FFN's up-projection step

All buffers are calloc'd (zero-initialized) rather than malloc'd. This ensures clean starting state for debugging and prevents mysterious failures from uninitialized memory.