College Basketball Backtest
The test we set for the model before running it, on every game of last season, and how it did.
- Research forecast. This model has not passed the tests we set for it before launch, so it makes no picks and shows no EV.
- It failed one of six tests we set before running it: the middle-50% range for game totals caught 46.9% of games against a 47.0% minimum.
- Team-level forecast: no roster, injury or minutes adjustments yet.
- We post the night before tip and again in the morning. The record tracks whether the line moves toward our number after we post, shown whether that is good or bad.
- Median of sportsbook lines collected before the forecast. No book is named and no price is shown.
- Games later in the week are posted as soon as a market lists them and are re-forecast at every post until tip, as results come in. The record scores each lead time separately.
Gate G-B0
Did not pass5 of 6 criteria passed2025-26 D-I v D-I games with a valid result: 5,752 games, walk-forward, run Nov 7, 3:42 AM ET. The criteria and their bounds were written down before the run, and the run counts once.
| # | Criterion | Result | Bound | Outcome |
|---|---|---|---|---|
| 1 | tests incl. leakage controls | 33 passed (ROADMAP G-B0 result) | all pass | Pass |
| 2 | log loss below Elo | ours 0.5314 against 0.5551 difference −0.0237 (95% interval −0.0295 to −0.0178) | ours lower, interval below zero | Pass |
| 3 | log loss below constant | ours 0.5314 against 0.6569 difference −0.1256 (95% interval −0.1369 to −0.1138) | ours lower, interval below zero | Pass |
| 4 | calibration slope and intercept | slope 0.95, intercept −0.0290 | slope 0.90 to 1.10, intercept −0.05 to 0.05 | Pass |
| 5 | interval coverage 50/80/95, margin and total | 50.0% range: margin 49.0%, total 46.9% 80.0% range: margin 79.7%, total 78.6% 95.0% range: margin 94.6%, total 94.7% | 50.0%: 47.0% to 53.0%; 80.0%: 77.0% to 83.0%; 95.0%: 93.0% to 97.0% | Fail |
| 6 | overtime rate within 1.5 points | predicted 5.8%, actual 5.7% | within 1.5 points | Pass |
Market comparison (reported, not gated)
Reported, not gated. The market line is the median of sportsbook lines (untimed); metrics only.
| Metric | Ours | Market line | Ours minus market |
|---|---|---|---|
| Log loss (win) | 0.5312 | 0.5208 | 0.0104 (95% interval 0.0045 to 0.0163) |
| CRPS, margin | 6.57 | 6.39 | |
| CRPS, total | 9.69 | 9.48 | |
| Mean absolute error, margin | 9.26 | 9.01 | |
| Mean absolute error, total | 13.71 | 13.37 |
5,747 games. Lower is better on every row.