Suppose you brought all the smartest baseball kids on the block together, locked them in a house for two weeks with an unlimited supply of Twinkies and beer, and asked them to pool all their statistical tools to predict the upcoming baseball season? What might the result look like?
SBNation’s Jason Brannon did that - well, he didn’t actually lock them up, but he combined (or averaged) several of the most egg-headed systems from the 2011 season. and compared to the actual results. The predictors he used were PECOTA (developed by Nate Silver for Baseball Prospectus), Marcel (developed by Jack Sackman and Tom Tango and named for a monkey named Marcel), CAIRO (an advanced version of Marcel) and BP-cofounder Clay Davenport’s DTs (for Davenport Translations, which are so complicated you’ll have to Google him and figure it out yourself).
...But the experts didn’t project a single exact win total for any team; the closest was the one-game difference between the Pirates, projected to win 71 and coming up with 72, and only three teams - the Braves, Dodgers and Giants—within two.
The worst predictions were Minnesota - winning 63 games against the projected 84, though we must cut the prognosticators some slack as the Twins’ two superstars, Joe Mauer and Justin Morneau, were out for most of the season - and Arizona , 73 actual vs. 94 projected, which I would file under the heading of “Who cares”?
I’d say this proves one thing about any system based on sabermetrics: they do a much better job predicting the past than the future.