# MLflow GenAI Evaluation Dashboard (Streamlit + Plotly)

## PURPOSE
The second "moment in the loop" in the Agent Bricks demo is `mlflow.genai.evaluate`:
one evaluation run scores a whole set of questions across three assessments —
relevance, safety, and a custom plain-English metric called "stays on catalog."
Relevance and safety come back at 100%, but "stays on catalog" is 80%, failing
on a question that asked for the capital of France. This project is a Streamlit
dashboard that visualises exactly that: per-metric pass rates, the failing
questions, and the same scores tracked over successive runs to catch a quiet drop.

## WHY THIS MATTERS
The demo's punchline is that "this gap over here is not a bug — that's the
evaluation earning its keep. This is the difference between hoping your agent
behaves and knowing it does." The capital-of-France example is the teaching
moment: relevance and safety each measure one thing and both pass, while the
custom guardrail correctly flags that the answer wandered off the product
catalog. A dashboard makes that distinction visible instead of a single number.

## HOW TO BUILD
1. Load evaluation runs from a JSON/CSV file shaped like an MLflow eval export:
   one row per question with `relevance`, `safety`, `stays_on_catalog`.
2. Build a Streamlit app with two tabs: **Run** (current scores) and
   **Trends** (scores across runs).
3. Use Plotly for a grouped bar chart of metric pass rates and a line chart of
   each metric over run index.
4. Add a table of failing questions, highlighting where relevance passes but
   stays_on_catalog fails — the exact contradiction the video explains.

Key snippet — surfacing the contradiction:

```python
df["off_catalog"] = (df["relevance"] == 1.0) & (df["stays_on_catalog"] == 0.0)
drift = df[df["off_catalog"]]
st.metric("stays_on_catalog", f"{df['stays_on_catalog'].mean():.0%}")
st.dataframe(drift[["question", "relevance", "safety", "stays_on_catalog"]])
```

## WHEN TO USE IN PRODUCTION
Use it as the human-facing companion to MLflow evaluation: reviewers see which
guardrail failed and on what question before promoting a version. It works
identically in development and production because "the exact same scores I'm
running here in development also run in production." Do NOT use it as the
evaluation engine or the serving path — it visualises results MLflow produced.
Not for a single question; it is built for a set.

## HOW IT HELPS STUDENTS
Students learn Streamlit layout, Plotly chart types, and how to communicate
eval results to a non-technical reviewer. They practise the demo's mental model
that different metrics measure different things, and they build the habit of
tracking scores over time rather than trusting one healthy run.

## STARTER CODE
See `starter/main.py` for a runnable Streamlit app with deterministic sample
eval runs, `# TODO` markers for file upload and MLflow API loading.
