# Ryzen AI Halo Model-Fit & Token-Speed API

## PURPOSE
A FastAPI service that answers the two questions this video keeps returning to: does a
local AI model fit on a machine, and how fast will it generate? You register machines
(AMD Ryzen AI Max 395 with 128 GB, the new 495 "Gorgon Halo" with 192 GB, Nvidia DGX
Spark, Apple Mac Studio M5 Ultra) and models (Qwen 3 235B at roughly 142 GB, GPT-OSS
120B, Kimi K2.5), and the API returns a fit verdict plus a bandwidth-bound tokens/sec
ceiling derived from the "pipe" math the presenter explains.

## WHY THIS MATTERS
The transcript's central idea is that generation speed depends on how wide the
memory-to-chip pipe is (256 GB/s on the 395 vs 273 GB/s on the 495, about 7% wider) and
how much of the model must pass through that pipe for each token. Extra memory does not
add speed; it adds room to load a model that simply does not fit into 128 GB. This
project turns that mental model into executable logic so students stop confusing capacity
with throughput.

## HOW TO BUILD
Use FastAPI with Pydantic request/response models and an in-memory catalog of machine and
model profiles (swap for SQLite later). The core is a pure function that checks GPU
allocation and computes the ceiling:

```python
def fit_and_ceiling(model_gb: float, machine: dict) -> dict:
    fits = model_gb <= machine["gpu_alloc_gb"]
    ceiling = machine["bandwidth_gbps"] / model_gb  # tokens/sec upper bound
    return {"fits": fits, "ceiling_tps": round(ceiling, 2)}
```

Expose `GET /machines`, `GET /models`, and `POST /estimate`. Keep the estimator honest by
labelling every response as a *calculation*, not a measurement, mirroring the video's
insistence on tagging numbers as vendor-claim, measurement, or math.

## WHEN TO USE IN PRODUCTION
Use it as a pre-purchase sizing check: an internal tool that tells a team whether a
candidate machine can load a client model before money is spent. Use it in a provisioning
pipeline to reject "it fits on paper but only using total RAM" assumptions. Do NOT use it
as a benchmark: it returns ceilings, not measured tokens/sec, and ignores KV-cache growth
for long conversations. For real latency SLAs, replace the estimator with recorded
LM Studio / llama.cpp measurements.

## HOW IT HELPS STUDENTS
Students learn to model capacity vs bandwidth, to expose a small calculator as a typed
HTTP API, and to be explicit about the provenance of a number. They practise the exact
reasoning the video applies to the 120-160 GB "window" only the 495 can run.

## STARTER CODE
See `starter/main.py` for a runnable FastAPI skeleton with the catalog, the
`fit_and_ceiling` estimator, real endpoints, and `# TODO` markers for persistence and
measured-latency integration.
