Page rendered in 0.3736 seconds
41 querie(s) executed
— Where BTF's Members Investigate the Grand Old Game
Tuesday, April 09, 2002
2001 Projections ? A Look Back; Projections Part I
Whose crystal ball is best?
It?s time once again to take a look at how the various projections did for the 2001 Season. In 2000, I looked at a bunch of various projection systems and how well they did in projecting the actual results of the various players. I looked at the top 170 players in terms of plate appearances (with some exceptions) and how each system did in projecting OPS. The results of that exercise can be found here.
Anyway, I?ve decided to do the same for the 2001 season only I?ll limit the analysis to myself, STATS, Inc. and Baseball Prospectus. The first thing I?ll do is simply give you the results for the three for the top 170 players in Plate Appearances:
Corr. = Correlation Coefficient
Anyhow, before anybody gets out of ahead of me, I?d like to talk about one of my favorite subjects: multiple endpoints. While generally multiple endpoints arguments occur when someone looks at a data set and then draws the restrictions to achieve a particular result, sometimes multiple endpoints becomes an issue even when no intent to rig the numbers is present.
This has happened here. If I extend the restrictions to the top 176 players, STATS winds up with a higher correlation coefficient than me. In other words, the two systems were about equal in terms of their ability to project the OPS of the players. This was essentially the same case as the year before as well.
I also wanted
to take a look at Baseball Prospectus? numbers. Last year BP?s projections
did every bit as well as mine and STATS and it was surprising that they not
only did worse than mine and STATS this year, but also worse than they had
done the year before. The first thing I checked was the
I eliminated the four
thing worth looking at, are the players that were missed by the greatest amount.
The five players with the largest percentage miss by my system were: Brady
Anderson, Cal Ripken, Jason Kendall, Edgardo Alfonzo and Barry Bonds. For
STATS it was: Bonds,
What about players projected using Minor League numbers? Well I really don?t know how and when STATS or BP used minor league stats, but there were 13 players out of the 170 who had less than 502 Major League appearances going into 2001. The correlation coefficients for these 13 were .60, .53 and .41 for me, STATS and BP respectively. While that looks much lower than overall, remember this is a very small sample and more importantly the range in performance for these 13 was much smaller than the overall range, and that drives those numbers down. The MAE (.037, .053, .051) and the RMSE (.045, .063, .068) are smaller than the overall? numbers, but this again is due to the smaller range. The MAPE (5.0%, 7.3%, 6.8%) is probably the most appropriate metric here, and for it, the numbers are roughly the same.
It should be noted however that the argument can be made that this represents a selective sample of only the minor leaguers who racked up a bunch of at bats, and that could affect the results. This is undoubtedly true, and this combined with the small sample makes the above numbers regarding minor leaguers not very meaningful. I?m going to try and do a more exhaustive look with next year?s stats.
One final note is that the problem from last year of underprojecting the group as a whole did not happen this year. The average OPS of the 170 was .814 and myself, STATS and BP had average OPS numbers for the group of .817, .814 and .820 respectively.
Projecting player performance is a critical aspect of player analysis, possibly the most critical. In the coming weeks I?ll discuss why that it is, what the difficulties in projecting are, and ways future projections might be able to be improved.
You must be logged in to view your Bookmarks.
Loser Scores 2015
(12 - 2:28pm, Nov 17)
Loser Scores 2014
(8 - 2:36pm, Nov 15)
Winning Pitcher: Bumgarner....er, Affeldt
(43 - 8:29am, Nov 05)
Last: ERROR---Jolly Old St. Nick
What do you do with Deacon White?
(17 - 12:12pm, Dec 23)
Last: Alex King
(15 - 12:05am, Oct 18)
Nine (Year) Men Out: Free El Duque!
(67 - 10:46am, May 09)
Who is Shyam Das?
(4 - 7:52pm, Feb 23)
Last: RoyalsRetro (AG#1F)
Greg Spira, RIP
(45 - 9:22pm, Jan 09)
Last: Jonathan Spira
Northern California Symposium on Statistics and Operations Research in Sports, October 16, 2010
(5 - 12:50am, Sep 18)
Mike Morgan, the Nexus of the Baseball Universe?
(37 - 12:33pm, Jun 23)
Last: The Keith Law Blog Blah Blah (battlekow)
Sabermetrics, Scouting, and the Science of Baseball – May 21 and 22, 2011
(2 - 8:03pm, May 16)
Last: Diamond Research
Retrosheet Semi-Annual Site Update!
(4 - 3:07pm, Nov 18)
What Might Work in the World Series, 2010 Edition
(5 - 2:27pm, Nov 12)
Last: fra paolo
Predicting the 2010 Playoffs
(11 - 5:21pm, Oct 20)
SABR 40: Impressions of a First-Time Attendee
(5 - 11:12pm, Aug 19)
Last: Joe Bivens, Minor Genius