Code · architecture · study mode

Inference simulation

Qwen3 Pure C
inference, end to end.

Qwen3 runs on your Mac, Windows, or Linux laptop — no cloud, no GPU, no account. This walkthrough traces the C code that makes it happen: how the model loads, how attention works, and why decode is GEMV, not GEMM.

The skill you're building: LLM Inference Engineering — earned across three acts.

⬡run_viz.c◷10 stages＄runs on your laptop
// walkthrough

Pick a stage

01

1 · Blueprint

Config, weights, RunState, and Transformer structs — the blueprint before any bytes load.

start here
02

2 · Load weights

mmap the GGUF file, skip the header, and walk pointers into TransformerWeights.

after stage 01
03

3 · Buffers

malloc_run_state — scratch activations and the KV cache sized from Config.

after stage 02
04

4 · Forward pass

rmsnorm, matmul, attention, RoPE, SwiGLU FFN — one token through all layers.

after stage 03
05

5 · Tokenizer

BPE vocab, merges, byte map — text to token IDs and back.

after stage 04
06

6 · Generate loop

sample(), chat REPL, main() — prefill, decode, and CLI flags.

after stage 05
07

7 · GEMV, not GEMM

Why matmul() is matrix × vector during decode — seven GEMVs per layer, and when it becomes GEMM.

after stage 06
08

8 · Every op a kernel

forward() as a launch schedule — calloc vs cudaMalloc, ~336 kernels/token, and two PCIe transfers.

after stage 07
09

9 · Attention & memory

Q·Kᵀ/√d, softmax, weighted values — correct math, but memory-bound (why FlashAttention exists).

after stage 08
10

10 · Prefill vs decode

One branch splits reading the prompt from generating — and why production batches prefill into GEMM.

after stage 09