Baseball for the Thinking Fan

You are here > Home > Primate Studies > Discussion
Primate Studies
— Where BTF's Members Investigate the Grand Old Game

## Monday, July 30, 2001

#### SUPER-LWTS ? A Player Evaluation Formula for the New Millennium

A top-notch but little-known sabermetrician presents his new player evaluation method.

Until now, the only rigorous metric for evaluating the complete   (defense and offense, including SB/CS) performance of a player was Total Baseball?s   TPR (Total Player Rating).? According to the Second Edition of Total   Baseball, TPR is defined as the sum of a player?s adjusted (for his home   park) Batting Runs, Fielding Runs, and Stolen Base Runs, minus his positional   adjustment, all divided by the Runs Per Win factor for that year (usually around   10 runs).? Presumably, TPR allows us to compare players across different   leagues, teams, eras, and defensive positions.?

Before I point out some of the weaknesses and limitations of TPR, and why super-lwts   is more comprehensive, let me discuss the components of TPR and what exactly   they are used to measure.? Like TPR, super-lwts is based on a linear   model of player evaluation.? In fact, super-lwts essentially expands   upon Pete Palmer?s original offensive linear weights formula.? Accordingly,   I would be remiss if I did not give Pete and John Thorn (coauthors of The   Hidden Game of Baseball) credit for providing the inspiration and basis   for the super-lwts formula.

A linear weights evaluation formula can be classified as a metric that expresses   a player?s offensive or defensive performance in runs above or below zero, where   zero is defined as the measure of an average player.? An offensive   linear weights formula, like Palmer?s classic one, represents a player?s theoretical   run contribution to an average team within his league and year(s).? In TPR,   offensive linear weights are park adjusted (and then converted into theoretical   wins or losses).? Without a park adjustment, a player?s offensive lwts   represents his hypothetical run contribution to an average team that plays   its home games in a particular park - namely that player?s home park.? With   a park adjustment, we can approximate a player?s theoretical run contribution   to an average team in an average park.? Whether and how to park adjust   a player?s stats is a complex and controversial subject in and of itself.? Without   addressing the nuances of park adjustment formulas, suffice it to say that,   in a perfect (park adjusted) world, park adjusting a player?s offensive lwts   allows us to fairly compare two players in the same league and year - but on   different teams.? Park adjusting also helps us to fairly compare   the performance of one player to another from a different year and/or   league.

The offensive lwts component (park adjusted or not) of TPR is, in my humble   opinion, one of the wonders of the world.? In fact, I use Palmer?s original   formula as one component of super-lwts, with only two minor adjustments:?   First, I use the current values for each of the offensive event coefficients   - calculated from a computer analysis of recent (1998-2000) play-by-play data.?   Second, I incorporate the SB/CS data in the offensive lwts formula rather   than adding it as a separate component.? (The results are the same, of   course, whether you use SB/CS data in the offensive lwts formula, as   I do, or add it in later, as TPR does.)

There are several other quantifiable aspects of offensive performance that   are missing in TPR.? One example is outs on base (OOB).? While the team   version of Palmer?s offensive lwts uses an OOB term, the individual version,   used to evaluate players, does not.? An OOB, for an individual player, is essentially   the same thing as a CS, but nowhere in TPR is this event recognized.??   Keep in mind that at this time I am defining an OOB by an individual player   as an out made by a player trying to stretch a single into a double or a   double into a triple (or the rare case of a triple into an inside-the-park   home run).? (I address baserunner, as opposed to batter-runner,   OOB in another super-lwts component.)? If these (batter-runner) OOB are   ignored by a lwts formula (which they are in TPR), then any player who has a   higher than average number of OOB will be offensively overvalued, and vice versa   (undervalued) for the cautious or ?efficiently aggressive? (lower than average)   batter-runner.? The reason why OOB for individual players are not included in   a player (as opposed to team) offensive lwts formula is probably because the   data are not readily available.? Because super-lwts makes use of play-by-play   data, it includes OOB (batter-runner and baserunner; they are contained   in two separate components) for individual player offensive ratings.? Since,   as I said, an OOB is essentially the same as a CS, it has a value of around   -.5 runs.

Another weakness, although easily corrected, of most offensive lwts formulas   that include SB/CS data, and of TPR, is that the values of the SB and CS are   too large, and the ratio of one to the other is incorrect.? In The Hidden   Game, Palmer and Thorn decided to arbitrarily inflate both values in order   to account for the presumed fact that stolen bases are attempted more   often when they are most valuable.? For example, while a SB in a late-inning,   10 to 0 blowout may have a run expectancy the same as in any inning and   at any score, it has almost no value in terms of win expectancy (it does   not significantly change either team?s chance of winning the game).? On the   other hand, a stolen base in the bottom of the 9th, with 2 outs and   the score tied, has a greater win expectancy than the run expectancy would ordinarily   suggest.? While this may have seemed like a brilliant supposition on Palmer?s   part (actually, I think he gave the credit to someone else), unfortunately,   many years after The Hidden Game was printed, Pete, myself, and probably   several other researchers, found that stolen base attempts were essentially   randomly distributed throughout a game.? In other words, they are not   attempted significantly more often when they are most valuable, such as during   the late innings of a close game.? As far as I know, Pete has never gone on   record to repudiate this presumption.? In any case, the correct values for a   SB and CS are closer to .19 and .46, respectively (in the modern era), than   the original .3 and .6, which are still used in TPR.? In the super-lwts   formula, the correct (above) values are used.

The last component (other than positional adjustment) of TPR is fielding   runs (defensive lwts).? Without getting into too much detail, suffice it   to say that the fielding runs component of TPR is fraught with all kinds of   accuracy and reliability problems.? Basically it is a very rough attempt   at quantifying, in terms of runs above or below average, a player?s fielding   contribution (compared to an average player at his position), based on   his putouts, assists, errors, and double plays.? You can look up the various   formulas (there are several, depending upon the position) for calculating a   player?s fielding runs.? A few weaknesses and limitations of TPR?s fielding   runs formulas are: 1) putouts at second base by a shortstop or second baseman   require little if any skill ? yet they are included in the formulas;   2) double plays are overvalued; 3) outfield assists have little meaning without   including ?hold percentage? (I will address this in more detail later); 3) for   some reason, outfield double plays are added to putouts (I suppose the   justification is that they tend to occur more often on difficult catches); 4)   defensive park factors can significantly affect outfield putout numbers (e.g.   leftfield in Fenway Park), and most importantly; 5) the various fielding   runs formulas (i.e. putouts, assists, and errors) do not account for the variations   in how many balls (per inning) are hit near (i.e. potentially catchable)   a player, due to the nature of the pitching staff (L/R, ground ball/fly ball,   power/finesse, etc.), or to plain old luck.? As you will see, Ultimate Zone   Rating (UZR), one of the components of super-lwts, does account for most   of these things (or at least does a pretty good job).? Like the offensive portion   of TPR, fielding runs can be computed using a player?s traditionally available   fielding stats, while calculating UZR requires detailed (hit-type and location)   play-by-play data.

The last part of the TPR formula (before converting runs into wins ? which   is trivial) is positional adjustment.? Basically all that a positional   adjustment does is add or subtract from a player?s pre-adjusted TPR, the average   TPR (also pre-adjusted, of course), in runs, of an average player at that position.?   Presumably, this puts all players, regardless of their defensive position, on   a level playing field (no pun intended).? For example, if, in 1998, Barry Larkin   had an unadjusted (for position) TPR of 25 runs in 155 games, and, also in 1998,   the average shortstop in the NL had a TPR of ?7 runs per 155 games, then Larkin   would have an adjusted TPR of 25 plus 7, or 32 runs.? As you can see,   what a positional adjustment really tells you are how many runs a player   is ?worth? above or below an average player (not a replacement player)   at that position.?

In my opinion, including a positional adjustment in a player?s TPR, particularly   without giving the adjustment ?factors? for each position, can be a bit misleading.??   Personally, I would rather know a player?s unadjusted TPR and the average   TPR at each position.? I can then do a positional adjustment or not - at my   own discretion.? The super-lwts formula and sub-formulas (the components)   do not include any kind of positional adjustments.? I do present the averages   at each defensive position, and the reader may, of course, use these in any   way that he or she wishes.

Next up from Mitchel   Lichtman ? the super-lwts formulas

Mitchel Lichtman Posted: July 30, 2001 at 06:00 AM | 3 comment(s) Login to Bookmark
Related News:

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

1.  Posted: July 31, 2001 at 12:10 AM (#604047)

“Personally, I would rather know a player?s unadjusted TPR and the average TPR at each position.  I can then do a positional adjustment or not - at my own discretion.”

By the same line of thinking, wouldn’t SUPER-LWTS be more flexible if you set the formula to runs rather than runs above average? For those of us who prefer using some type of replacement level as the baseline, this design choice makes the formula more flexible and useful.

Like the positional adjustment, people could then make adjustments at their own discretion.

2. Mitchel Lichtman Posted: July 31, 2001 at 12:10 AM (#604049)

Jim, valid point, although, unlike positional adjustments, anyone can make the “runs above/below average to runs” adjustment, without any additional information.

The rest of the article (detailed explanation of the SUPER-LWTS components and the player lists) will be forthcoming (next week?)...

3. Tangotiger Posted: July 31, 2001 at 12:10 AM (#604050)

I’m pretty sure that Mickey will be giving us his ranking based on /500 PA or something.  Therefore, it should be a trivial matter to convert that against a “replacement level” baseline, of whatever you choose.

You must be Registered and Logged In to post comments.

<< Back to main

Thanks to