Baseball for the Thinking Fan

You are here > Home > Primate Studies > Discussion
Primate Studies
— Where BTF's Members Investigate the Grand Old Game

## Tuesday, December 18, 2001

#### Total Baseball’s Fielding Runs?A Primer

How are Pete Palmer’s fielding runs calculated?

This article is part of a broader series of articles, written for SABR-L [The Society of Baseball Research’s Mailing list] and crossposted to STATLG-L [Baseball (and Lesser Sports) Discussion List ], comparing the results generated by different defensive measurement systems. The players chosen for this comparison were two 1978 shortstops later traded for each other - San Diego’s Ozzie Smith and St. Louis’s Garry Templeton. I was intrigued by the similarities in their overall fielding stats prior to the trade and the separation that developed very quickly after they were swapped for each other. I’ve shortened the article from the original post but left the details of the method intact.

In developing Total Player Rating (TPR), Pete Palmer devised a measurement   of fielding skill in terms of runs. Fielding Runs (FR) is added to runs generated   from batting skill and baserunning (stealing) skill (and pitching skill for   pitchers) to come up with an overall total rating expressed in runs, later converted   to wins.

Palmer calculates FR for SS (see Note 1) by first calculating a league average   for the position according to the following formula:

AVG(POS,LG) = .20*(PO+2A-E+DP)/(league PO - league K)

where PO, A, E, and DP are the total putouts, assists, errors, and double plays   for the league’s shortstops, and league PO - league K represents an estimate   of the total number of outs made by the league’s fielders. Palmer multiplies   assists by 2 because "more fielding skill is generally required to get   one than to record a putout".

Once Palmer has that average for the position and league, he compares what   a SS actually did produce to what he estimates an average player would have   roduced in the same number of outs:

FR = .20*(PO+2A-E+DP) - ((team PO - team K) * AVG(POS,LG) * %PT)

where PO, A, E, and DP are the putouts, assists, errors, and double plays registered   by the shortstop, team PO - team K is the number of outs made by that team’s   fielders, and %PT is an estimate of the amount of playing time at the position   by this shortstop, based upon the player’s complete fielding line and the number   of plate appearances. Palmer doesn’t publish his formula for estimating playing   time, making it impossible to reproduce his results. In my calculations below   I use actual playing time calculated from Retrosheet data to demonstrate the method.

NL SS, in 1978, had 3191 PO, 6199 A, 332 E, and 1105 DP. The league as a whole   had 52,000 PO and 9905 K. The AVG(POS,LG) is therefore

.20*(3191+2*6199-332+1105)/(52000-9905) = 0.0777

Garry Templeton played 1353 2/3 of the 1437 2/3 innings played by the Cardinals   (0.942 of the total). Templeton had 285 PO, 523 A, 40 E, and 108 DP. The Cardinals   had 4313 putouts as a team and fanned 859 hitters. Templeton’s FR (see Note   2) are:

.20*(285+2*523-40+108)-((4313-859)*.0777*.942) = 27

Ozzie Smith played 1327 of the 1433 2/3 innings played by the Padres (0.926   of the total). Ozzie had 264 PO, 548 A, 25 E,
and 98 DP. The Padres had 4301 putouts as a team and fanned 744 hitters. Ozzie’s   FR are:

.20*(264+2*548-25+98)-((4301-744)*.0777*.926) = 31

TB7 has Templeton at 23, Ozzie at 26.

The biggest flaw in FR is that Palmer makes no effort to derive event weights   from the actual relationship between the defensive events and their impact on   run scoring, as he does with his offensive measures. An assist is worth two   times a putout based on Palmer’s subjective assessment of its value. Palmer   also subtracts errors from plays made, even though not every error adds a baserunner,   and adds DPs to the total even though DPs are also counted in PO and A.

FR is flawed in three important ways:

• There is no obvious relationship between a defensive event and its effect   on run scoring. Palmer presents no evidence that an infield assist is twice   as valuable in preventing runs as is a putout, or that an average error costs   as much as an average putout, or that the player making a DP pivot (on which   he gets both a putout and an assist) should be credited with as much value   as a player who makes four putouts.
•
• Palmer’s estimates of playing time overestimate the playing time of regular   players and underestimate the playing time of substitutes, artificially depressing   the FR for the regulars at a position.
•
• Palmer makes no adjustment for differences in the distribution of balls   in play from team to team. Ground ball/fly ball tendencies of pitching staffs   and the LH/RH distribution of hitters faced by a team do change the distribution,   and the variations from team to team are not minimal (see Note 3).

The first flaw above, in my opinion, is the primary reason why most analysts   discredit FR. It’s very natural for an analyst, when presented with a formula   that looks like:

R = AW1+BW2+CW3

to ask how the developer of the formula came up with the weights. If the developer   of the formula doesn’t have satisfactory answers, as Palmer does not, the credibility   of the entire formula is questioned, especially when the results aren’t always   intuitive and the developer of the formula acts as though the results are just   as accurate as the results from other, more rigorously developed formulae that   measure other aspects of R.

Notes:

1. The formula used here applies only to 2B, 3B, and SS. Palmer has different   formulae for the other positions, but the concept for calculating FR is the   same.
2.
3. These are rounded to the nearest whole number.
4.
5. There are those who argue that a pitcher "pitches to his fielders",   e.g. pitches in such a way as to control the direction of the ball in play,   and therefore a fielder’s ability might be responsible for generating those   extra opportunities. I don’t see any evidence in play-by-play data that such   an effect occurs on a regular basis, so
while I suppose it’s possible I think it’s mostly independent of fielding   skill, and thus it is appropriate to account for differences in ball in play   distribution when evaluating fielders.

Mike Emeigh Posted: December 18, 2001 at 05:00 AM | 7 comment(s) Login to Bookmark
Related News:

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

1. Charles Saeger Posted: December 18, 2001 at 12:18 AM (#604539)
Mike, you missed the most important flaw.

When evaluated as a team, each team is pretty much equal. There are a few differences, like errors and assist rates, but every team makes 27 putouts a game. FR for a team vary by only a few a year.

Palmer ignored the hits, the team failures. Were he to modify FR even just to make the denominator (PO-SO+H-HR), he would greatly improve his accuracy, even without all the silly weights. That way, good defensive teams would have better FR totals than bad defensive teams.
2. J. Lowenstein Apathy Club Posted: December 18, 2001 at 12:18 AM (#604542)
Charles, there would be an added problem in making the team totals differ in any substantial way. It creates a "double-count" problem for pitchers (since they are given all the credit for defensive events in Palmer's system), making the team LWts come out strangely... good defensive teams get better, and bad defensive teams get worse. As it is, at least the team totals are consistent.

The problem could also be fixed (somewhat) by removing some of the credit a pitcher gets for certain defensive events.
3.  Posted: December 18, 2001 at 12:19 AM (#604557)
Craig's point is correct. Palmer didn't include hits in FR because they are already included in his pitching runs totals, and the intent was for the individual events to be additive. That's not so much a flaw in FR as it is a choice on Palmer's part as to how to assign the responsibility for hits - and traditionally, 100% of the responsibility for hits has been assigned to the pitchers.

Speaking of responsibility for hits:

It is my opinion that there are only two ways that one can legitimately make this assignment without resorting to a guess:

1. The pitchers are 100% responsible for all hits;
2. The pitchers are only responsible for hits that leave the ballpark, and the fielders are 100% responsible for all other hits.

-- MWE
4. Alan Shank Posted: December 19, 2001 at 12:19 AM (#604572)
"It is my opinion that there are only two ways that one can legitimately make this assignment without resorting to a guess:

1. The pitchers are 100% responsible for all hits; 2. The pitchers are only responsible for hits that leave the ballpark, and the fielders are 100% responsible for all other hits."

The trouble is, neither of those is true. The biggest problem in evaluating fielding, now that play-by-play, direction-and-distance data is available, is to divvy up responsibility for all those events where the ball is put in play, but not out of the park.
Cheers,
Alan Shank

5.  Posted: December 19, 2001 at 12:19 AM (#604581)
Alan:

Certainly, the availability of "play-by-play, direction-and-distance data" could make it possible to "divvy up the responsibility". But I don't think we need to do that. My belief - which I think is supported by a lot of evidence, including analysis of available play-by-play data, Voros's work, the tendency of pitchers to sustain success more readily when they restrict the number of balls put into play against them, the existence of what James called the "Tommy John" class of pitchers - is that the pitcher's level of control over the results of what happens when a ball is put into play within the ballpark is small enough so that we can effectively ignore it, and treat the fielders as being 100% responsible for the outcome when a ball is put into play within the ballpark.

I think we can gain a lot more by proceeding down that path than by expending a lot of energy trying to figure out something that we will find it extremely difficult to validate.

-- MWE
6. Alan Shank Posted: December 21, 2001 at 12:19 AM (#604601)
I really don't believe that the pitcher has so little effect on the outcome of balls put in play that it can be ignored. The difficulty of studying this issue is that is requires multiple-year, batter-vs-pitcher data that is difficult to compile. If we could get about three years' worth of batter-vs-pitcher data, then split it up into pitcher-only (K, BB, HBP, HR) vs. ball-in-play (everything else) and plunk it into the XR formula, maybe we'd get closer to an answer to this. My guess is that:

1. the ball-in-play outcomes are more random than the pitcher-only
2. they are obviously affected, to a considerable extent, by the quality of fielding
3. we would still see pitcher influence, in that the results for a given team would vary by pitcher by more than a random amount

We'd want to compare Boston's ball-in-play data with different pitchers, inlcuding Pedro, etc.

Cheers,
Alan
7. Alan Shank Posted: December 21, 2001 at 12:19 AM (#604602)
After re-reading Voros' article, I had another thought. If a pitcher has some control over how hard the ball is hit, this would likely show up more in the numbers of doubles and triples than in all hits. That's why I want to use xRuns, so that the value of the hits is considered as well as the number. We would be looking at xRuns/PA.
Cheers,
Alan

You must be Registered and Logged In to post comments.

<< Back to main

### Support BBTF

Thanks to
Sheer Tim Foli
for his generous support.

### Bookmarks

You must be logged in to view your Bookmarks.

### Syndicate

Page rendered in 0.2767 seconds
42 querie(s) executed