Three Things To Know About Preseason Projections

Several college basketball analysts recently released preseason projections for the upcoming season. How well do these models forecast the final conference standings? Here are some guidelines based on recent history:

1. The top statistical models correctly predict ~25% of teams’ conference finishes exactly, and ~50% within one position.

CBB_Preseason_Projections_2014-15

I studied the prediction models of Ken Pomeroy, Dan Hanner and TeamRankings. The three systems rely on many of the same inputs, so perhaps it’s no surprise that they perform similarly well. In 2014-15, all three models correctly predicted the exact finish of 24-29% of teams (including ties), and called 49-50% within one slot.

CBB_Preseason_Projections_accuracy

Using the more rigorous metric of conference wins and losses, Pomeroy found a slight edge for Hanner and TeamRankings’ methods over his own. All three forecasted every team’s conference record with an average error of about 2.2 games, which is rather impressive.

2. The preseason coach and media polls predict conference standings about as well as statistical models.

CBB_Preseason_Projections_Polls_2014-15

Like statistical models, the league polls were exactly correct for about one-quarter of teams, and within one place for half. Even though coaches and writers usually take much different approaches than the statistical models, they end up with similar accuracy.

CBB_Preseason_Projections_Polls_accuracy

Polls were substantially more accurate than projections as of about four years ago, but the methods have recently converged. You can pick your favorite of several possible explanations, such as: 1) Projections are improving, and will surpass polls in the future; 2) Projections are improving, and pollsters are learning from their models, so they will improve in tandem; 3) Both projections and polls are about as good as they’ll ever be, and any further movements are just noise.

3. 2014-15 was a bad year for both projections and polls.

For several years, the models and polls each had an average error of about 1.8 places in their conference standings projections. Last year, the average error jumped to 2.1.

CBB_Preseason_Projections_Polls_history

There was some consolidation across leagues last year, which skews that statistic a bit (the expected error is greater when a conference has more teams). But across metrics and systems, 2014-15 was much harder to predict than 2013-14:

CBB_Preseason_Projections_Polls_history_2

I wouldn’t read too much into a one-year trend — sometimes unlikely events happen, making good forecasts look bad. We saw unusual patterns in last season’s Patriot League, to pick one example close to our Big Apple hearts. (The largest individual error in most predictions came from Davidson, which won the Atlantic 10 after being picked anywhere from ninth to 13th.)

But if the projections and polls have another rough year in 2015-16, it may be worth wondering if there are any inherent forces (transfers? rule changes?) making college basketball less predictable.

==

*Note: Conference USA is excluded from all data because it has not sponsored a preseason poll in the past, and independents are excluded for obvious reasons. For leagues whose polls were separated by division — such as the MAC and OVC — each division is treated as its own league. Where both coach and media polls were available, the coaches’ poll was used. Many historical projections obtained via MasseyRatings.com.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s