# FreshnessGuard: Postgres-to-Warehouse Streaming Freshness API

## PURPOSE
Artie streams data out of Postgres into Snowflake, Databricks and Iceberg in near
real time — "sub-minute," as Jacqueline Cheong puts it. Her blunt framing in this
interview is the whole point of this project: *"if data is late, it's simply just
wrong."* FreshnessGuard is a FastAPI service that tracks per-table replication lag,
scores it against a sub-minute freshness SLO, and exposes it over HTTP so that
fraud models, AI support agents and analytics teams can see staleness *before* they
act on stale rows.

## WHY THIS MATTERS
The video walks through why real-time replication is unforgiving: a fintech fraud
model on Databricks that learns a transaction an hour late raises mean-time-to-
detection; an AI agent that thinks a booking hasn't happened yet will create a
duplicate ticket. Data arriving late is not a performance nit — it is a correctness
bug. FreshnessGuard teaches you to treat lag as a first-class SLO you can observe.

## HOW TO BUILD
1. Model each replicated table as a source with `last_event_at` and an SLO
   (`slo_seconds`, default 60).
2. Store an in-memory ring buffer of `(table, observed_at, lag_seconds)` samples.
3. Expose `GET /freshness/{table}` returning lag, SLO status and a burn-rate flag.
4. Add `GET /health` and `POST /samples` so a CDC consumer can push heartbeats.
5. Add a background task that marks a table `STALE`/`BREACH` as lag grows.

```python
@app.get("/freshness/{table}")
def freshness(table: str):
    lag = _lag_seconds(table)          # now - last_event_at
    st = _status(lag, SLO_SECONDS)     # OK | WARN | BREACH
    return {"table": table, "lag_seconds": lag, "status": st}
```

## WHEN TO USE IN PRODUCTION
Use it when downstream consumers make decisions on replicated data faster than the
batch window — fraud scoring, inventory, AI agents, executive dashboards. Wire the
`BREACH` status into alerts and into agent pre-flight checks ("don't answer from this
table yet"). Do NOT use it as a data mover; it observes lag, it does not replicate.
It is the wrong tool for overnight batch warehouses where an hour-old table is fine,
and it is not a substitute for the replication engine itself.

## HOW IT HELPS STUDENTS
You practice FastAPI routing, Pydantic schemas, background tasks and SLO thinking —
the exact observability muscle that real streaming teams (Netflix, DoorDash,
Instacart, and Artie itself) build internally. You learn to reason about lag as a
correctness property, not just a latency number.

## STARTER CODE
See `starter/main.py` for a runnable skeleton with an in-memory lag store, a
deterministic stub clock, SLO evaluation and `# TODO` markers for the CDC heartbeat
endpoint and the alerting hook.
