Building a Prediction Model in Public: B1 to B13

Predictium's model has been through 13 named iterations since late December 2025. One was struck from the record entirely. Several made things worse. A few changed everything. This is the full account — what worked, what didn't, and the numbers at every step.

The first model was deliberately simple: Ridge regression for spreads, XGBoost for totals, roughly 107 features. Team ratings, pace, efficiency metrics, some early player tracking data. Nothing exotic. The goal was to establish a baseline we could measure everything against.

Walk-forward validation across three seasons. Those were the numbers to beat — and for a while, beating them proved harder than we expected.

New Year's Eve, 2025. We added four player delta features designed to capture how tonight's lineup differs from the expected rotation. When a starter sits, their minutes redistribute across the roster, and the delta quantifies the net impact.

The initial numbers looked spectacular: spread R² leapt to 0.1788. We nearly shipped it.

Then we found the bug. The function determining player availability only checked who played in historical games. Players who were OUT never appeared in box scores, so they were tagged "UNKNOWN" and assigned 80% of their normal minutes instead of zero. Every delta in the training set was compressed into a narrow band of -3.5 to +3.5 — the real range was -120 to +65. The model was learning from a whisper of the actual signal.

After the fix:

That +52% on totals remains the single largest improvement in the model's history. Most of it came from exponentially-weighted moving averages at multiple half-lives, not the deltas themselves. But the delta bug established a rule we still follow: if your numbers look too good, find the lie before you celebrate.

RAPM — Regularized Adjusted Plus-Minus — is the gold standard for isolating individual player impact. It solves a ridge regression over play-by-play data to decompose every player's contribution into offense, defense, and pace. We computed it across 166 biweekly snapshots spanning five seasons.

B3 added 21 RAPM roster features: for each game, weight every player's RAPM ratings by their projected minutes to produce a portrait of tonight's lineup quality — baseline, projected, and delta values for offense, defense, and net.

Underwhelming in aggregate. But buried in the results was a signal that would take months to properly exploit: on high-disruption games — where significant roster changes were in play — spread R² jumped from 0.1125 to 0.1311. A 16.5% improvement, concentrated exactly where it matters most.

Sixteen travel features: flight distance, timezone crossings, cumulative fatigue load, cross-country trips, jet lag indicators, schedule density. The split-head architecture was formalized here — separate models for spreads and totals, each with its own feature pipeline.

Travel helped spreads, hurt totals. This was the first clear evidence of something we'd eventually build the entire architecture around: spreads and totals want fundamentally different things from the feature set.

B5 never shipped to production. It was pure research. And it produced the most important architectural insight in the model's history.

The idea was deceptively simple: instead of predicting the game total directly, predict each team's score using a single model trained on "canonical" data. Every game contributes two rows — one from each team's offensive perspective. This doubles the training signal and teaches the model how teams score irrespective of home/away position.

We tested ten configurations:

Two other discoveries fell out of the research: away scores are consistently more predictable than home scores across every configuration, and linear models match XGBoost for totals — the non-linear signal simply isn't there.

If the ElasticNet partially captures RAPM signal but doesn't fully absorb it, there should be residual information in the prediction errors that correlates with RAPM values. B6 tested this by adding a post-prediction calibration layer — a linear correction of +0.87 points per unit of net RAPM delta for spreads, with a split offensive/defensive adjustment for totals.

Barely moved the needle on spreads. Actually hurt totals. On paper, this looked like a failed experiment. But the calibration layer would prove its value once combined with later improvements. It remains in production through B13 — a reminder that components which seem useless in isolation can matter in ensemble.

B7 changed zero features. Not a single new column in the training data. All it changed was the regularization: Ridge (α=1.0) became ElasticNet (α=0.1, L1=0.9) for spreads, with similar retuning for totals.

Nineteen percent. From hyperparameters.

L1 regularization zeros out noise features that Ridge dutifully fits. The model had been memorizing weak signals for months, and we'd been adding features on top of the overfitting, trying to compensate with complexity for what we should have fixed with discipline. Sometimes the biggest wins come from removing what isn't working.

With the regularization foundation solid, B8 added 13 lineup disruption features built entirely from box score history: the impact magnitude of absent players, the quality gap to their replacements, binary flags for top-player absences, and rotation shuffle metrics. No RAPM dependency — these features work across all five seasons.

ElasticNet's L1 immediately pruned six of the thirteen. The surviving seven earned their place.

Strong spread improvement. But totals collapsed. The disruption features were tuned for the spread head's aggressive L1=0.9, and the total head was running the same regime. Totals needed something different — we just hadn't figured out what yet.

B9 exists because of a single insight: spreads and totals don't just want different features — they want opposite regularization.

Spreads thrive under aggressive L1 (0.9). A few strong predictors dominate, and everything else is noise. Prune ruthlessly. Totals thrive under Ridge-like regularization (L1=0.1). The signal is distributed — many features contributing small amounts. Keep them all alive and let the ensemble of weak signals do its work.

B9 also added three matchup interaction features for the total head: a harmonic mean of scoring averages, an analytical expected total derived from pace and efficiency, and a combined expected points metric.

This was the architecture that ran in production for months.

We found a serious flaw in our RAPM decomposition. The system separating each player's impact into offense, defense, and pace was using correlated priors — player-level O/D correlation was 0.9999. It couldn't distinguish a 30-point scorer who can't guard a chair from a two-way All-Star.

The fix (V3 RAPM) introduced independent priors. O/D correlation dropped to 0.14. We regenerated all 166 snapshots and retrained.

B10 showed spread R² of 0.2003 and total R² of 0.1054. The numbers were beautiful.

They were also wrong. The 2025-26 RAPM data had scrambled lineup-to-team assignments from a bug in our play-by-play provider. The ElasticNet, doing exactly what it's designed to do, latched onto these pseudo-random features as signal. B10's improvement was an artifact of corrupted data.

We struck B10 from the record. If you can't trust the data, you can't trust the numbers.

B11 is B10 retrained on fully corrected V3 RAPM. Same 204 features. Same architecture. Just clean data.

Spread R² dropped, and it should have. B9's RAPM features were carrying noise that happened to correlate with outcomes. With clean data, L1 prunes most RAPM features because they're redundant — net rating and RAPM correlate at r=0.44, and the model already has net rating.

Totals improved because the Ridge-like total head (L1=0.1) keeps features alive. Clean RAPM contributes where dirty RAPM added noise.

B11 became the honest baseline. Everything after it is real.

If RAPM features get pruned inside the ElasticNet, maybe they shouldn't live there at all. Maybe they need their own model.

B12 tested this on totals: a standalone RAPM Ridge model (25 features — 21 roster projections plus 4 engineered composites) blended 80/20 with the ElasticNet. Total R² improved by +0.0065.

When we tried the same thing for spreads, it barely moved. Then we found the problem: 135 games across the 2022-25 seasons had zero RAPM features due to missing snapshot coverage. The blend optimizer was learning that RAPM contributes nothing, because for those specific games, it literally did. Once we rebuilt the RAPM data to fill coverage gaps, the spread blend flipped from negligible to significant.

B13's final architecture:

• Head A: ElasticNet (204 features, α=0.1, L1=0.9)
• Head B: RAPM-only Ridge (25 features, α=1.0)
• Spread blend: 57% ElasticNet / 43% RAPM
• Total blend: 77% ElasticNet / 23% RAPM
• Plus: B6 post-prediction calibration on top

The RAPM head is weak alone — spread R² of 0.1196, total R² of just 0.006. But blended, it fills gaps the primary model can't. RAPM captures roster quality changes instantly. When a player is traded or hurt, RAPM adjusts on the first game. Rolling team statistics take 10–15 games to reflect the same change. During that window, the ElasticNet is working with stale data. The RAPM head covers the gap.

We tested conditional blending — adjusting the weight based on disruption level — but it added only +0.0001 R². A fixed blend is effectively optimal. The model already knows when RAPM matters; we don't need to tell it twice.

B10 was struck for corrupted RAPM data. Its numbers were inflated by scrambled lineup assignments from an upstream provider bug.

Not every experiment earns a version number. Some earn a headstone.

• Shot quality (xPPS): Four separate approaches — team-level, player-level, roster-based redistribution, matchup interactions. We built a logistic regression xPPS model on 945,000 shots. Perfectly calibrated. Completely useless for prediction. The existing features — offensive rating, effective FG% — already encode everything shot quality would add.
• Referee features: Scraped assignment data across five seasons. Built rolling referee tendency profiles. Neutral impact. Vegas already prices this in.
• Opponent-quality interactions: Slightly hurt performance. The model handles quality differentials through team ratings without needing explicit interaction terms.
• Volatility features (12 engineered): XGBoost assigned zero importance to all twelve. The model sees no signal here. Neither do we.
• Tanking indicators: Negligible. RAPM and net rating already encode team quality decline implicitly.
• Player impact as post-prediction adjustment: The signal drowns in noise at the individual game level. Player impact works as a pre-prediction feature — let the model learn the coefficient — not as an after-the-fact correction.

We mention these not as a disclaimer but as evidence. A model that only reports its wins is a model you can't trust with your money.

B13 explains roughly 21% of the variance in NBA game margins. The remaining 79% is the game itself — the missed free throw, the bad bounce, the call that could have gone either way.

Twenty-one percent doesn't sound like much. But in a market where most public models explain 8–12%, it's a meaningful edge — especially when concentrated in the high-disruption games where the market is slowest to adjust.

Playoff-specific models are next on the research list. Different dynamics when rotations tighten, preparation deepens, and teams face the same opponent seven times in two weeks.

We'll keep building in public. The model gets better when we're honest about where it's weak.