You are here > Home > Primate Studies > Discussion
 
Primate Studies — Where BTF's Members Investigate the Grand Old Game Monday, July 30, 2001SUPERLWTS ? A Player Evaluation Formula for the New MillenniumA topnotch but littleknown sabermetrician presents his new player evaluation method. Until now, the only rigorous metric for evaluating the complete (defense and offense, including SB/CS) performance of a player was Total Baseball?s TPR (Total Player Rating).? According to the Second Edition of Total Baseball, TPR is defined as the sum of a player?s adjusted (for his home park) Batting Runs, Fielding Runs, and Stolen Base Runs, minus his positional adjustment, all divided by the Runs Per Win factor for that year (usually around 10 runs).? Presumably, TPR allows us to compare players across different leagues, teams, eras, and defensive positions.? Before I point out some of the weaknesses and limitations of TPR, and why superlwts is more comprehensive, let me discuss the components of TPR and what exactly they are used to measure.? Like TPR, superlwts is based on a linear model of player evaluation.? In fact, superlwts essentially expands upon Pete Palmer?s original offensive linear weights formula.? Accordingly, I would be remiss if I did not give Pete and John Thorn (coauthors of The Hidden Game of Baseball) credit for providing the inspiration and basis for the superlwts formula. A linear weights evaluation formula can be classified as a metric that expresses a player?s offensive or defensive performance in runs above or below zero, where zero is defined as the measure of an average player.? An offensive linear weights formula, like Palmer?s classic one, represents a player?s theoretical run contribution to an average team within his league and year(s).? In TPR, offensive linear weights are park adjusted (and then converted into theoretical wins or losses).? Without a park adjustment, a player?s offensive lwts represents his hypothetical run contribution to an average team that plays its home games in a particular park  namely that player?s home park.? With a park adjustment, we can approximate a player?s theoretical run contribution to an average team in an average park.? Whether and how to park adjust a player?s stats is a complex and controversial subject in and of itself.? Without addressing the nuances of park adjustment formulas, suffice it to say that, in a perfect (park adjusted) world, park adjusting a player?s offensive lwts allows us to fairly compare two players in the same league and year  but on different teams.? Park adjusting also helps us to fairly compare the performance of one player to another from a different year and/or league. The offensive lwts component (park adjusted or not) of TPR is, in my humble opinion, one of the wonders of the world.? In fact, I use Palmer?s original formula as one component of superlwts, with only two minor adjustments:? First, I use the current values for each of the offensive event coefficients  calculated from a computer analysis of recent (19982000) playbyplay data.? Second, I incorporate the SB/CS data in the offensive lwts formula rather than adding it as a separate component.? (The results are the same, of course, whether you use SB/CS data in the offensive lwts formula, as I do, or add it in later, as TPR does.) There are several other quantifiable aspects of offensive performance that are missing in TPR.? One example is outs on base (OOB).? While the team version of Palmer?s offensive lwts uses an OOB term, the individual version, used to evaluate players, does not.? An OOB, for an individual player, is essentially the same thing as a CS, but nowhere in TPR is this event recognized.?? Keep in mind that at this time I am defining an OOB by an individual player as an out made by a player trying to stretch a single into a double or a double into a triple (or the rare case of a triple into an insidethepark home run).? (I address baserunner, as opposed to batterrunner, OOB in another superlwts component.)? If these (batterrunner) OOB are ignored by a lwts formula (which they are in TPR), then any player who has a higher than average number of OOB will be offensively overvalued, and vice versa (undervalued) for the cautious or ?efficiently aggressive? (lower than average) batterrunner.? The reason why OOB for individual players are not included in a player (as opposed to team) offensive lwts formula is probably because the data are not readily available.? Because superlwts makes use of playbyplay data, it includes OOB (batterrunner and baserunner; they are contained in two separate components) for individual player offensive ratings.? Since, as I said, an OOB is essentially the same as a CS, it has a value of around .5 runs. Another weakness, although easily corrected, of most offensive lwts formulas that include SB/CS data, and of TPR, is that the values of the SB and CS are too large, and the ratio of one to the other is incorrect.? In The Hidden Game, Palmer and Thorn decided to arbitrarily inflate both values in order to account for the presumed fact that stolen bases are attempted more often when they are most valuable.? For example, while a SB in a lateinning, 10 to 0 blowout may have a run expectancy the same as in any inning and at any score, it has almost no value in terms of win expectancy (it does not significantly change either team?s chance of winning the game).? On the other hand, a stolen base in the bottom of the 9^{th}, with 2 outs and the score tied, has a greater win expectancy than the run expectancy would ordinarily suggest.? While this may have seemed like a brilliant supposition on Palmer?s part (actually, I think he gave the credit to someone else), unfortunately, many years after The Hidden Game was printed, Pete, myself, and probably several other researchers, found that stolen base attempts were essentially randomly distributed throughout a game.? In other words, they are not attempted significantly more often when they are most valuable, such as during the late innings of a close game.? As far as I know, Pete has never gone on record to repudiate this presumption.? In any case, the correct values for a SB and CS are closer to .19 and .46, respectively (in the modern era), than the original .3 and .6, which are still used in TPR.? In the superlwts formula, the correct (above) values are used. The last component (other than positional adjustment) of TPR is fielding runs (defensive lwts).? Without getting into too much detail, suffice it to say that the fielding runs component of TPR is fraught with all kinds of accuracy and reliability problems.? Basically it is a very rough attempt at quantifying, in terms of runs above or below average, a player?s fielding contribution (compared to an average player at his position), based on his putouts, assists, errors, and double plays.? You can look up the various formulas (there are several, depending upon the position) for calculating a player?s fielding runs.? A few weaknesses and limitations of TPR?s fielding runs formulas are: 1) putouts at second base by a shortstop or second baseman require little if any skill ? yet they are included in the formulas; 2) double plays are overvalued; 3) outfield assists have little meaning without including ?hold percentage? (I will address this in more detail later); 3) for some reason, outfield double plays are added to putouts (I suppose the justification is that they tend to occur more often on difficult catches); 4) defensive park factors can significantly affect outfield putout numbers (e.g. leftfield in Fenway Park), and most importantly; 5) the various fielding runs formulas (i.e. putouts, assists, and errors) do not account for the variations in how many balls (per inning) are hit near (i.e. potentially catchable) a player, due to the nature of the pitching staff (L/R, ground ball/fly ball, power/finesse, etc.), or to plain old luck.? As you will see, Ultimate Zone Rating (UZR), one of the components of superlwts, does account for most of these things (or at least does a pretty good job).? Like the offensive portion of TPR, fielding runs can be computed using a player?s traditionally available fielding stats, while calculating UZR requires detailed (hittype and location) playbyplay data. The last part of the TPR formula (before converting runs into wins ? which is trivial) is positional adjustment.? Basically all that a positional adjustment does is add or subtract from a player?s preadjusted TPR, the average TPR (also preadjusted, of course), in runs, of an average player at that position.? Presumably, this puts all players, regardless of their defensive position, on a level playing field (no pun intended).? For example, if, in 1998, Barry Larkin had an unadjusted (for position) TPR of 25 runs in 155 games, and, also in 1998, the average shortstop in the NL had a TPR of ?7 runs per 155 games, then Larkin would have an adjusted TPR of 25 plus 7, or 32 runs.? As you can see, what a positional adjustment really tells you are how many runs a player is ?worth? above or below an average player (not a replacement player) at that position.? In my opinion, including a positional adjustment in a player?s TPR, particularly without giving the adjustment ?factors? for each position, can be a bit misleading.?? Personally, I would rather know a player?s unadjusted TPR and the average TPR at each position.? I can then do a positional adjustment or not  at my own discretion.? The superlwts formula and subformulas (the components) do not include any kind of positional adjustments.? I do present the averages at each defensive position, and the reader may, of course, use these in any way that he or she wishes. Next up from Mitchel Lichtman ? the superlwts formulas

BookmarksYou must be logged in to view your Bookmarks. Hot TopicsWhat do you do with Deacon White?
(17  1:12pm, Dec 23) Last: Alex King Loser Scores (15  12:05am, Oct 18) Last: mkt42 Nine (Year) Men Out: Free El Duque! (67  10:46am, May 09) Last: DanG Who is Shyam Das? (4  8:52pm, Feb 23) Last: RoyalsRetro (AG#1F) Greg Spira, RIP (45  10:22pm, Jan 09) Last: Jonathan Spira Northern California Symposium on Statistics and Operations Research in Sports, October 16, 2010 (5  12:50am, Sep 18) Last: balamar Mike Morgan, the Nexus of the Baseball Universe? (37  12:33pm, Jun 23) Last: The Keith Law Blog Blah Blah (battlekow) Sabermetrics, Scouting, and the Science of Baseball – May 21 and 22, 2011 (2  8:03pm, May 16) Last: Diamond Research Retrosheet SemiAnnual Site Update! (4  4:07pm, Nov 18) Last: Sweatpants What Might Work in the World Series, 2010 Edition (5  3:27pm, Nov 12) Last: fra paolo Predicting the 2010 Playoffs (11  5:21pm, Oct 20) Last: TomH SABR 40: Impressions of a FirstTime Attendee (5  11:12pm, Aug 19) Last: Joe Bivens, Minor Genius St. Louis Cardinals Midseason Report (12  12:42am, Aug 10) Last: bjhanke Napoleon Lajoie: Definition of Grace (9  12:38am, Jul 01) Last: Hang down your head, Tom Foley Youth Baseball Hitting Drills: Shine the Light (5  6:47am, Mar 11) Last: Pat Rapper's Delight 

Page rendered in 0.1782 seconds 
Reader Comments and Retorts
Go to end of page
Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.
1. Jim Furtado Posted: July 31, 2001 at 12:10 AM (#604047)“Personally, I would rather know a player?s unadjusted TPR and the average TPR at each position. I can then do a positional adjustment or not  at my own discretion.”
By the same line of thinking, wouldn’t SUPERLWTS be more flexible if you set the formula to runs rather than runs above average? For those of us who prefer using some type of replacement level as the baseline, this design choice makes the formula more flexible and useful.
Like the positional adjustment, people could then make adjustments at their own discretion.
Jim, valid point, although, unlike positional adjustments, anyone can make the “runs above/below average to runs” adjustment, without any additional information.
The rest of the article (detailed explanation of the SUPERLWTS components and the player lists) will be forthcoming (next week?)...
I’m pretty sure that Mickey will be giving us his ranking based on /500 PA or something. Therefore, it should be a trivial matter to convert that against a “replacement level” baseline, of whatever you choose.
You must be Registered and Logged In to post comments.
<< Back to main