# Prefill vs Generation Token Pipeline Simulator

## PURPOSE
A SimPy + NumPy discrete-event simulation of a local language model serving requests, built
directly on the video's two key words: **prefill** (the model reading your question) and
**generation** (the model writing the answer). It models a 9,000-token prompt like the one
Micro Center used, then simulates how long a user waits for the first token (TTFT, driven
by prefill compute) and how fast text scrolls afterward (driven by memory bandwidth — the
"pipe"), for the AMD 395 (256 GB/s) versus 495 (273 GB/s).

## WHY THIS MATTERS
The transcript's whole argument is that these two phases scale differently. Generation is
bounded by pipe width divided by model bytes per token, so the 495 gets at most ~7% faster
and Micro Center measured only 2-4%. Prefill depends on raw compute, which is where Nvidia's
DGX Spark separates itself (~1,700 vs ~340 tokens/sec on the same model, about 5x). This
simulator lets students *see* that trade-off instead of reading a spec table.

## HOW TO BUILD
Model the model as an entity: prefill takes `prompt_tokens / prefill_rate` seconds, then
each output token takes `model_gb / bandwidth_gbps` seconds. Use SimPy processes and a
NumPy random stream for prompt-length jitter:

```python
def serve(env, model, machine, prompt_tokens):
    ttft = prompt_tokens / machine["prefill_tps"]
    yield env.timeout(ttft)
    per_token = model["size_gb"] / machine["bandwidth_gbps"]
    for _ in range(model["out_tokens"]):
        yield env.timeout(per_token)
```

Run many trials, collect TTFT and generation-rate arrays, and print medians (the video
stresses median-of-repeats, not best case).

## WHEN TO USE IN PRODUCTION
Use it for capacity planning and UX budgets: does a chat agent's time-to-first-token meet
target under a given prompt length, and will scrolling stay faster than a human reads
(~50 tokens/sec)? Use it to decide whether a workload is prefill-heavy (long documents,
coding agents) or generation-heavy (short chat). Do NOT use it as a performance benchmark
— it is a model, calibrated with assumed rates, not hardware measurements. Do NOT trust it
for multi-user contention without adding a scheduler.

## HOW IT HELPS STUDENTS
Students learn discrete-event simulation, queue/process modelling, and Monte Carlo medians.
More importantly they internalize why bandwidth and compute point at different hardware —
the video's central analytical lesson.

## STARTER CODE
See `starter/main.py`: a full SimPy model with machine profiles, per-request processes,
NumPy jitter, and a median summary across runs, plus `# TODO` markers for batch scheduling
and measured-rate calibration.
