# AI Video Benchmark CLI — Prompt Test-Bench Analyzer

## PURPOSE
A Click + Pandas command-line tool that ingests the raw results of an AI video
model test suite and emits a ranked comparison table. In the source video the
host runs a fixed battery of tests against Kling 4.0 and Seedance 2.5 — text-to-
video, image-to-video, character consistency (a scar that must stay on the left
eye), physics (glass shatter, a collapsing brick tower), lip-sync, dancing, and
censorship probes (Tom Cruise, Mickey Mouse, Donald Trump) — recording the very
first generation, not a cherry-picked best. This CLI turns those rows into cost-
per-clip, quality, and pass-rate analytics.

## WHY THIS MATTERS
The transcript's method is the lesson: judge models on *consistent, first-try*
results across standardised tests, and always weigh score against price (Kling 4
at ~$0.80 vs Seedance 2.5 at ~$2.85 for a 5s 1080p clip — about 70% cheaper).
Doing that by hand invites bias. A pandas pipeline makes the comparison
reproducible and lets you slice by test category.

## HOW TO BUILD
1. `pip install click pandas`
2. Define a CSV schema: `test_id, model, category, prompt, score, cost_usd`.
3. `load()` reads with pandas; `summarise()` groups by model and category and
   computes mean score, mean cost, and score-per-dollar.
4. Expose `report`, `compare`, and `cost` subcommands via Click.

```python
@click.group()
def cli():
    """Analyze AI video model benchmark results."""

@cli.command()
@click.argument("csv_path", type=click.Path(exists=True))
@click.option("--model", default=None)
def report(csv_path, model):
    df = load(csv_path)
    if model:
        df = df[df["model"] == model]
    click.echo(summarise(df).to_string(index=False))
```

## WHEN TO USE IN PRODUCTION
Use it before signing a model contract, when migrating a pipeline from one
vendor to another, or to run monthly regression on outputs as models change.
Not a good fit for one-off ad-hoc spot checks, for qualitative judging that
genuinely needs a human eye, or when your sample size per test is tiny.

## HOW IT HELPS STUDENTS
Students learn pragmatically-typed Click CLIs, tidy-data modelling, groupby
aggregation, and the discipline of comparing tools on measured evidence rather
than vibes — a transferable skill far beyond video generation.

## STARTER CODE
`starter/main.py` ships a Click CLI with a synthetic seed dataset, a `summarise`
function, and `# TODO` markers for CSV loading and export.
