# Frac Telemetry Stream Simulator (SimPy + NumPy)

## PURPOSE
A discrete-event simulation of the streaming telemetry pipeline the transcript describes: hydraulic fracturing jobs each emit ~100 GB per day to the cloud, with ~100 jobs running worldwide. This simulator models jobs as producers, an ingestion/agent consumer with finite throughput, and a queue between them. It answers the operational question the video raises implicitly — how many agents (or how much ingestion capacity) are needed so the backlog does not grow without bound?

## WHY THIS MATTERS
The video's core argument is capacity: "It is impossible for human beings to consume data at this scale. And so, you need agents of some sort to do this." It also notes the internal agent took eight months to build before tooling like Genie made it available out of the box. This project lets students experiment with the throughput math behind that decision — producer rate versus consumer rate — and see queue growth, backlog, and utilization emerge from first principles.

## HOW TO BUILD
1. Use NumPy to generate a per-job daily data rate and jitter (deterministic seed).
2. Use SimPy `Environment` with a bounded `Resource` for the ingestion/agent pool.
3. Model each job as a process that emits chunks at a steady rate into a `Store` (queue).
4. Model the agent as a process draining the queue at a fixed throughput.
5. Collect time series (queue depth, utilization) and print summary statistics.

Key snippet from `starter/main.py`:
```python
def agent(env, queue, pool, throughput_gb_s, stats):
    while True:
        chunk = yield queue.get()
        with pool.request() as req:
            yield req
            service_time = chunk / throughput_gb_s
            yield env.timeout(service_time)
            stats["processed_gb"] += chunk
```
Run with `python starter/main.py --jobs 100 --gb-per-job 100 --agent-throughput 8`.

## WHEN TO USE IN PRODUCTION
Use it for capacity planning before buying ingestion clusters, for choosing how many agents to run against a telemetry stream, or for teaching queueing intuition. Do NOT use it as a live autoscaler or as a substitute for real load testing — it is a what-if model with synthetic rates, not a monitor.

## HOW IT HELPS STUDENTS
Students learn SimPy process modeling, NumPy distributions, and Little's Law-style reasoning about queues. They see why "just add more jobs" breaks a pipeline and why throughput headroom is an engineering decision, not an afterthought.

## STARTER CODE
See `starter/main.py` — a deterministic SimPy simulation with a seeded NumPy rate generator, summary stats, and `# TODO` markers for retries and multi-agent pools.
