A model tested hard enough to catch its own leak.
Every chart here is real state DOT crash data or a real, spatially-cross-validated evaluation.
—
honest model accuracy
ROC AUC, spatial cross-validation, no leakage
—
average precision
—
accuracy
Real collisions, per state
Raw totals from each state's public DOT dataset — the source for every hotspot on the simulation map.
Per-year baseline the simulation is calibrated against.
| State | Real records | Years | Per year | Source |
|---|---|---|---|---|
| Loading… | ||||
Honest, not leaky
An earlier version scored 0.867 ROC AUC using cluster-derived features to predict cluster-derived labels — a leak. Spatial cross-validation, holding out entire geographic blocks, gives the honest number. Lower is correct here, not a regression.
Leaky vs. honest
Leaky Honest
Accuracy by state
What drives the prediction
Hour of day and month dominate — animal activity is strongly time-patterned. Species features carry near-zero importance; none of the five source datasets record a reliable species field.