Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Primate Studies > Discussion
Primate Studies
— Where BTF's Members Investigate the Grand Old Game

Thursday, November 21, 2002

And the Beat Goes On: Derek Jeter and the State of Fielding Analysis in Sabermetrics - Part 6

In part six of eight, Mike visits Zone Rating and related statistics.

Building a Better Mousetrap - Zone-Based Rating Methods (ZR
and UZR)

 

Bill James has made a number of contributions to the field of statistical analysis of baseball, but perhaps his greatest contribution is one
that often goes unacknowledged. In the 1984 Baseball Abstract, James issued
the following call to his readers:

“PROJECT SCORESHEET is an attempt to build a network of fans to collect ...
scoresheets, and to construct the necessary administrative framework to get
the scoresheets to the public. I’m asking for your help.” (Baseball
Abstract
, 1984 edition, page 251)

I have no idea whether James envisioned what would happen as a result of
this single call for help, but there is absolutely no question that Project
Scoresheet was an overwhelming success. The volunteers of Project Scoresheet
built an organization, for the first time, placed detailed play-by-play
accounts of major league baseball games into the hands of a public that was
hungry for them, and that could use them to take detailed looks at the game
beyond that which could be evaluated from the stat lines that appeared in the
newspapers and in the postseason trade books. Project Scoresheet begat STATS,
Inc. and the Baseball Workshop, and spwaned a new group of analysts who have
carried the field forward from the early days of James and Palmer.

Traditional defensive statistics provide a record of player successes,
some of which arise out of plays that players make themselves, and others
of which arise out of plays that are really initiated by other fielders.
Traditional defensive statistics also provide a partial record of
defensive failures, those which were (in the opinion of an official scorer)
sufficiently bad to warrant the recording of an error. Analysts have known
for years, however, that there are many balls on which outs could have been
recorded, and perhaps should have been recorded, that are scored as hits
because no one actually touched the ball, or because a ball fell into
“no-man’s land”. These balls are really, in the final analysis, failures on
the part of the defenders just as much as errors are - but traditional
defensive statistics don’t let the analyst know how to determine the
responsibility for these failures. If a player made 300 plays, it was
impossible to know whether he made those 300 plays while letting 50 others
go unmade, or 100, or 200. To properly evaluate fielding skill, it is
necessary to have a complete record not only of success, but also of
opportunities; that record has not been available for most of baseball
history.

As Project Scoresheet grew, and split into STATS and the Baseball Workshop,
and as more fully-scored games became available, analysts began thinking about
how best to determine defensive responsibility for balls in play. If while
scoring a game, the scorer could unambigously identify the type of ball that
was hit (GB, FB, line drive) and the location on the field where the ball
landed, it would then be possible to assign balls in play to a particular
fielder, based on assumptions about where fielders are normally positioned.
Fielders could then be evaluated by determining the number of balls that they
converted into outs, as a percentage of the number of balls assigned to them. If
shortstop A had 100 balls hit “into his zone”, and converted 85 of them into
outs, while shortstop B had 150 balls hit “into his zone” and converted 120 of
them into outs, then shortstop A, because he made a higher percentage of plays
per opportunity, would be considered to be a better fielder than shortstop B,
even though shortstop B made more plays overall and would likely show up as a
higher-rated fielder in a method like Fielding Runs.

In 1989, STATS Inc. published the initial set of Zone Ratings (ZR) for
fielders. The STATS scorers used a grid placed over the field that divided the
field into zones (see this link for the grid that STATS uses), and recorded the location
of every ball hit into play on that grid, recording the location where the
ball landed for fly balls and line drives, and recording the infield location
where either a ground ball was fielded, or through which it passed on its way
to the outfield. Zones were grouped and assigned to specific fielders, and
fielders were then rated based on the percentage of balls that they turned
into outs in their specific zones.

At roughly the same time, Pete DeCoursey and Sherri and Dave Nichols of the
Baseball Workshop published a similar approach, using a similar grid developed
for Project Scoresheet (see this link for that grid). The first set of ratings under this scheme,
called Defensive Average (DA), were published in Pete’s newsletter in early
1990 and later republished to the Usenet group rec.sport.baseball. DAs were
published regularly until 1996, but have not been published since then,
leaving the field of defensive analysis systems based on play-by-play data
pretty much to ZR-based systems. (I’m limiting the discussion here to systems
for which the underlying methodology is publicly available. There are other
PBP-based systems, such as those developed by Mike Gimbel and Tom Tippett, but
those are proprietary systems for which the full methodology is not publicly
available.) I’ve chosen not to discuss DA in this article, since the method is
no longer being used to the best of my knowledge, but I will note that the
DA analysis covered more of the field than did ZR and allowed for overlapping
coverage assignments where more than one fielder could make a play.

In developing ZR, STATS chose to exclude certain portions of the field from
consideration on the basis that balls hit into those areas of the field should
be considered to be “uncatchable”. Unlike DA, STATS also assigned zones to one
and only one fielder, with no overlapping responsibility; balls hit in-between
fielders were either assigned to one fielder or the other, or were included in
one of the “uncatchable” zones. STATS also chose to exclude infield air
outs from consideration when defining the rating for infielders, so that
infielders are rated only on ground balls. Bunts are also excluded. In their
original ZRs, STATS also gave infielders double credit for starting double
plays, but this was eliminated after 1999 (ZRs before that time that appear on
the Internet, such as those on the Fox Sports Web site, have been corrected).

In this
article
on the Diamond Mind Web site, Tom Tippett accurately summarizes
the basic issues with ZR as a measure of defensive ability:

“The first problem is that they don’t count all the balls. For example, no
infielder is charged with an opportunity when a grounder is hit down the lines,
in the holes, or up the middle. The only plays that go into the zone ratings
are the ones where the ball is hit more or less at a fielder. The ones that are
left out are the ones that only the best fielders get to. The net result is a
system that places a lot more emphasis on good hands than range.”

“The second problem occurs when an infielder starts a double play. STATS
credits him with two outs and one opportunity. ...” (This problem has been fixed
since Tom published the article.)

“The third problem is that errors are mixed in with the ability to get to the
ball in the first place. For example, in 1999, Edgardo Alfonzo had a zone rating
of .921, while the norm for his position was .905. At face value, you’d think
this means that he covered more ground than the average second baseman. But he
also made 8 fewer errors than the average second baseman, given the number of
chances he handled. If you change 8 of his outs into errors, his zone rating
drops to .902. Now he looks like a fielder with average range and very good
hands.”

For these reasons, Zone Rating is not much of an improvement over the
approaches based on analysis of traditional defensive statistics, and in fact
(as will be seen below) the ZR rankings are very similar to the rankings
achieved from using systems based on evaluation of traditional statistics.

The powers that be at STATS, well aware of these deficiencies with ZR, developed
Ultimate Zone Rating, first published in the STATS 2000 Baseball Scoreboard, as a
way to address these problems. Alan Shank provided a description of UZR on the Big Bad Baseball Web
site in April 2000:

“...what STATS did was to take each of their zones, break them up into 10-foot
sections and calculate the rate at which balls batted into each section were
converted to outs. Then each player’s record was calculated for each section.
For example: in zone Q (RCF), 340’ from home, a fly ball was turned into an
out 40% of the time. If a RF makes an out in that section, he gets .6 point;
if it falls in, he gets -.4 point. In this way, more difficult plays are
weighted higher. An average fielder would end up with zero points.  The
Ultimate Zone Rating is not a percentage, but a number of weighted outs
recorded above (or below) an average fielder.”

The STATS UZR generates outs above or below average. Mitchel Lichtman, aka MGL, has developed a method to convert UZR
outs saved into runs saved, and the data that I’m presenting for shortstops
here is taken from that article.

Derek Jeter fares better in UZR for the 1998-2000 seasons than he fares in
any of the methods that are not directly based on play-by-play. In Table 11, I
present the ZR and UZR for each of the regular AL shortstops during 1998-2000,
along with the now-familiar results from each of the other methods.

Table 11. ZR and UZR, AL Shortstops (800+ innings), 1998-2000

1998 Team Inn ZR UZR WSRAA FR DFT DWP SSF/9
Vizquel, O CLE 1316.0 0.881 23 5.98 7.44 11 0.541 4.08
Stocker, K TBA 940.0 0.841 21 1.12 11.63 14 0.540 4.41
Garciaparra, N BOS 1255.3 0.839 18 -4.47 -15.26 -11 0.478 3.84
Bordick, M BAL 1238.3 0.873 12 3.56 18.26 4 0.501 4.42
Jeter, D NYA 1304.7 0.855 8 -1.92 -20.02 -3 0.510 3.99
DiSarcina, G ANA 1370.7 0.831 2 0.05 -3.63 3 0.507 3.99
Rodriguez, A SEA 1389.3 0.863 -2 -1.67 2.06 0 0.488 4.35
Caruso, M CHA 1121.3 0.816 -8 -8.28 -7.33 -16 0.440 4.49
Meares, P MIN 1270.0 0.816 -9 -0.38 -7.68 0 0.492 4.11
Gonzalez, A TOR 1398.3 0.850 -12 5.61 -9.40 5 0.536 3.69
Cruz, D DET 1163.3 0.856 -13 -1.17 16.10 17 0.531 5.04
Tejada, M OAK 915.0 0.794 -20 -1.71 2.16 0 0.477 4.40
     
1999 Team Inn ZR UZR WSRAA FR DFT DWP SSF/9
Vizquel, O CLE 1214.3 0.862 27 -5.01 1.57 -7 0.487 4.09
Sanchez, R KCA 1128.7 0.889 24 5.09 31.94 24 0.552 4.82
Bordick, M BAL 1355.0 0.867 19 9.57 35.38 23 0.560 4.40
Cruz, D DET 1300.3 0.843 17 3.66 4.68 20 0.525 4.31
Batista, T TOR 860.7 0.870 9 4.53 10.18 13 0.523 4.28
Garciaparra, N BOS 1171.7 0.813 5 5.18 -7.53 0 0.513 4.04
Jeter, D NYA 1395.7 0.833 -1 -6.89 -33.55 -13 0.475 3.64
Tejada, M OAK 1377.3 0.830 -1 3.48 7.13 4 0.512 4.20
Guzman, C MIN 1069.0 0.782 -4 -1.84 -7.20 -3 0.493 4.18
Clayton, R TEX 1149.3 0.834 -4 -1.15 3.10 -7 0.487 4.64
Caruso, M CHA 1114.7 0.837 -13 -2.84 -19.92 -10 0.462 4.25
Rodriguez, A SEA 1114.7 0.843 -23 2.83 7.23 3 0.510 4.42
     
2000 Team Inn ZR UZR WSRAA FR DFT DWP SSF/9
Rodriguez, A SEA 1285.0 0.893 16 6.77 8.05 17 0.536 3.98
Sanchez, R KCA 1198.0 0.878 15 6.08 15.18 28 0.476 4.45
Martinez, F TBA 887.7 0.885 12 2.48 31.20 9 0.505 4.73
Garciaparra, N BOS 1185.0 0.862 8 2.16 -2.55 -4 0.491 4.37
Tejada, M OAK 1400.3 0.867 6 -0.38 3.17 -4 0.546 4.46
Clayton, R TEX 1237.0 0.858 5 -0.23 -2.59 -4 0.492 4.15
Guzman, C MIN 1307.0 0.806 2 2.00 -14.15 -9 0.456 3.78
Bordick, M BAL 865.0 0.841 0 -3.45 -14.38 -6 0.481 3.94
Jeter, D NYA 1278.7 0.811 -1 -12.81 -36.47 -27 0.490 3.50
Valentin, J CHA 1212.3 0.847 -5 5.55 20.59 0 0.495 4.30
Gonzalez, A TOR 1225.3 0.826 -5 -2.89 -6.82 16 0.487 4.36
Cruz, D DET 1355.3 0.830 -6 -0.51 3.95 13 0.548 4.34
Vizquel, O CLE 1328.7 0.838 -9 -3.96 -0.54 -4 0.491 3.98

The non-PBP based methods have fairly high positive
correlations with ZR (r=+0.576 for Palmer’s FR, r=+0.516 for DFTs, r=+0.475
for CAD DWP, r=+0.500 for WSRAA), and lesser positive correlations with UZR
runs (r=+0.338 for FR, r=+0.349 for DFTs, r=+0.354 for CAD DWP, r=+0.325 for
WSRAA). ZR and UZR correlate positively as well, as one might expect, although
not as closely as I expected (r=+0.582). There is a mild positive correlation
between ZR and SS fieldable opportunities (r=+0.327), and almost no
correlation between UZR and SS fieldable opportunities (r=+0.011).

The one major bias within zone-based ratings, to which I’ve alluded earlier
in this series, is the underlying assumption of a constant area of coverage
for a given fielding position; e.g. if the average shortstop gets to *x*
percent of balls in a particular zone, then a shortstop that exceeds *x* by
definition has saved his team outs and runs. But the area that a shortstop can
cover changes with positioning, and to the extent that the ball distribution
in play deviates from the norm, the shortstop’s positioning will deviate as
well, and the area of the field that he can be expected to cover will also
change.

One might think that positioning changes shouldn’t matter very much. But
there is evidence that indicates that it doesn’t take much of a change in
positioning to affect the ability of a fielder to make plays. In 1997, Don
Malcolm, assisted by some colleagues, undertook what is to date (as far as I
am aware) the only significant study of the ability of fielders to make plays
based upon the force with which the ball is hit, and the distance that the
fielder has to move in order to make the play. The Force and Distance
(FAD) method is described on page 508 of the 1998 Big Bad Baseball Annual:

“For each ball hit in and around the infield, then, we would record a
graded estimate of the Force of the batted ball, and the Distance the fielder
closes to the ball when it went through the infield had to travel in the
attempt to glove it. The rest of the system would record the same type of
information about the play as that which fueled zone rating and defensive
average - out or hit or error ... Two people were involved in the observations:
one measuring force, the other distance. At different games, they would trade
assignments. After sufficient training and sample games, a set of more than 40
games was scored using the FAD method in 1997 in Oakland, San Francisco, Los
Angeles, and San Diego.”

Force was rated on a scale of 1-5 (softest hit to hardest hit ball), with
half-steps allowed. Distance was measured in terms of how far a player had to
move in or back, and left or right, on a scale of 0-3, with half-steps on the
scale except between 0 and 1. The results came from a 10-game random sampling
of all games scored. When the infielder didn’t have to move for a ball in play
(distance=0), the rate at which the fielder converted the ball in play into an
out was 94% (regardless of how hard the ball was hit). When the fielder did
have to move (direction non-zero), the rate dropped to 57%. Looking at a
specific value of force (3.5 - slightly above-average force covering about 25%
of all of the balls put into play), the rate was 93% when the fielder didn’t
have to move, 29% when he did. Granting that this is a very small sample size
and covers only a few teams, the data does suggest that the fielder’s ability
to cover a particular section of the field depends a great deal on where he is
positioned before the play, because when he has to move at all his ability to
make a play is greatly reduced.

Although we really need more data than just this one limited set to draw
any sort of defnitive conclusion on fielder positioning, I think it’s
reasonable to draw an inference that teams will tend to position their
fielders in such a way as to maximize the likelihood of a ball being hit
directly at them (or close enough so that they won’t have to roam very
far in order to field it). Suppose we assume that the inference is valid,
and then take a look at some particulars of the ball in play distribution
against the Yankees to try to figure out where their fielders might be
positioned.

Start with the 2Bs. Table 12 shows the distribution of GB in play
around Yankee 2Bs, in the middle (M, z4M and z4MD), direct (D, z4 and z4D),
and hole (H, z34 and z34D) compared to the league distribution, for 1999 and
2000. I left the 1998 data out because of uncertainties in the ball
location information that were discovered by Chris Dial after he read Part 2
of this series. The distribution is presented for all hitters, and broken
down for RH and LH hitters.

Table 12. Distribution of Yankee and AL GBIP in 1999-2000, 2B Zones

All GBIP M D H %M %D %H
Yankees 268 531 376 22.8% 45.2% 32.0%
AL 4688 7133 5198 27.5% 41.9% 30.5%
             
LHB GBIP M D H %M %D %H
Yankees 139 341 265 18.7% 45.8% 35.6%
AL 2528 4650 3897 22.8% 42.0% 35.2%
             
RHB GBIP M D H %M %D %H
Yankees 129 190 111 30.0% 44.2% 25.8%
AL 2160 2483 1301 36.3% 41.8% 21.9%

Note that the distribution skews toward the 1B/2B hole
against the Yankees, when compared to the league average, for both RH hitters
and LH hitters. That would suggest that the Yankees would be more likely to
shade the 2B over toward 1B for RH hitters as well as for LH hitters.

Now let’s look at the 3B. Table 13 presents the same data for 3Bs, looking
at the direct (D, z5 and z5D) and hole (H, z56 and z56D).

Table 13. Distribution of Yankee and AL GBIP in 1999-2000, 3B Zones

All GBIP D H %D %H
Yankees 134 416 24.4% 75.6%
AL 2330 6628 26.0% 74.0%
         
LHB GBIP D H %D %H
Yankees 19 74 20.4% 79.6%
AL 252 1135 18.2% 81.8%
         
RHB GBIP D H %D %H
Yankees 115 342 25.2% 74.8%
AL 2078 5493 27.4% 72.6%

Here the distributional skews are a bit different. The distribution
skews toward the SS hole against RHB, and toward the line against LHB. The number of
balls hit down the 3B line by LHB is fairly small, however, and I would conclude
that with this small number of balls in that direction hit by LHB, any benefit the
Yankees would gain would be minimal. Thus, I would expect the Yankee 3B to be shading
toward the hole in either case. (This conclusion is supported by the fact that only
one of those 19 balls hit down the line by LHB was converted into an out.)

Finally, SS. In the absence of other information, one might assume that because
the 3Bs are likely to be shading toward the SS hole and the 2Bs are also shading
toward the SS hole, the SS are likely to be shading up the middle. A look at the
ball distribution around SS (Table 14) suggests other possibilities:

Table 14. Distribution of Yankee and AL GBIP in 1999-2000, SS Zones

All GBIP M D H %M %D %H
Yankees 377 337 416 33.4% 29.8% 36.8%
AL 6907 5426 6628 36.4% 28.6% 35.0%
             
LHB GBIP M D H %M %D %H
Yankees 128 81 74 45.2% 28.6% 26.1%
AL 2358 1275 1135 49.5% 26.7% 23.8%
             
RHB GBIP M D H %M %D %H
Yankees 249 256 342 29.4% 30.2% 40.4%
AL 4549 4151 5493 32.1% 29.2% 38.7%

As on the right side of the infield, the distribution around SS is
also skewed toward the SS hole. Now with the 3B likely already shading toward the
hole, one might expect the Yankees would think they had the hole well covered, and
shade Jeter toward the middle anyway. If the Yankees are doing this, then one might
also expect that, after you adjust for the balls fielded by the 3B in the SS
hole, Jeter would be turning a low percentage of balls in play into outs, since he
would be moving toward the hole from a position closer to the middle of the
infield. Table 15 shows what actually happened in these circumstances.

Table 15. Outs made by Yankee/AL SS in hole after removing 3B plays, 1999-2000

Mike Emeigh Posted: November 21, 2002 at 06:00 AM | 45 comment(s) Login to Bookmark
  Related News:

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. Mark Field Posted: November 21, 2002 at 02:05 AM (#607339)
Mike, are you trying to understand Jeter's ability or his </i>performance</i>? IOW, if you are trying to analyze Jeter in order to improve his performance in the future, then you'd want to have a better idea of his range/ability and shift his positioning to take full advantage of that. That's what I see you doing here.

OTOH, if Jeter is positioned wrongly, he hasn't made plays that a "better" shortstop (i.e., one positioned correctly) would have made. In this sense, why doesn't UZR give us an accurate evaluation of Jeter's past performance?
   2. tangotiger Posted: November 21, 2002 at 02:05 AM (#607340)
These articles just get better and better!

I suppose then, to improve UZR, you have to determine the actual ball distribution by specific pitcher and handedness of hitter. Then given this ball distribution, you should determine the optimal positioning for each of the fielders (don't know yet how to do that). Once you've got that, you need to determine the probable success rates for every zone (don't know yet how to do that), for each of these ball distributions. Once you've got that, you figure out how many total outs you'd get given this specific ball distribution based on pitcher/batter-handedness. And you compare this to the number of outs Jeter recorded.

Phew! That's alot of work. The question then is the extent to which the specific pitcher ball distribution is different from other pitchers. It will be very interesting to see what Jeter faces.

Good job Mike!
   3. Warren Posted: November 21, 2002 at 02:05 AM (#607345)
I remember that we had a discussion on Fanhome about how Jeter could have finished ahead of A-Rod, when it looked like A-Rod was better in each zone. It's possible that the number of balls in each zone would explain it (just like the "paradox" about how player A could have a better batting average than player B both before *and* after the all-star break, but still have a worse batting average for the year), but I'm not sure if that really was the answer - MGL said he'd take a look at the issue, but the thread kind of died as we moved on to other things. Was that issue ever resolved?
   4. Mike Emeigh Posted: November 21, 2002 at 02:05 AM (#607346)
Mike, are you trying to understand Jeter's ability or his performance?

I guess the best way to put this is that I'm trying to understand how well Jeter's ability is captured in the methods we have for interpreting his performance. IOW, is Jeter's defensive performance, as measured by (fill in the method), an accurate representation of his defensive ability?

OTOH, if Jeter is positioned wrongly, he hasn't made plays that a "better" shortstop (i.e., one positioned correctly) would have made. In this sense, why doesn't UZR give us an accurate evaluation of Jeter's past performance?

I'm not suggesting that Jeter is positioned *wrongly*, but that he is likely to be positioned differently than the normal SS, because of factors that have little to do with his ability. And if he is positioned differently, assuming (for the moment) average defensive skill, the proportion of balls that he will be able to field and turn into outs is likely to differ from the values calculated in UZR. In some areas, he should do better; in others, he should do worse.

Nelson Briles tells a story (for which I cannot vouch for the accuracy) about a game in which he was pitching for the Pirates with Orlando Cepeda on 2B. Briles looked up and saw that Bill Mazeroski was cheating toward the 2B bag, and Briles waved at him to move back toward the hole (which is where he expected the hitter to hit the pitch he was about to throw). Mazeroski inched a few steps in that direction, and Briles threw a pitch which wasn't put into play. Before the next pitch, Briles looked up again and saw Mazeroski cheating back up the middle again. This time, he went ahead and pitched. The hitter grounded a ball through the 2B hole, exactly where Briles had wanted Mazeroski to play - and Roberto Clemente threw out Cepeda trying to score. After the game, Mazeroski explained to Briles that he knew that the hitter was likely to hit the ball through the hole, but Maz also knew that Clemente was more likely to throw out a runner at the plate than the Bucs' CF could, and therefore he cheated up the middle to cut a ball off if it happened to be hit that way.

Briles's story illustrates an important principle: defensive positioning is a risk-reduction activity. The fielders want to be positioned in such a way as to maximize the probability that a ball in play will be turned into an out and to minimize the damage if it is not. That might mean that an individual fielder might not be positioned in the location where he, personally, can make the most plays (as I noted in the article, it's a reasonable assumption that Jeter is not positioned in a location where he can make the most plays). Rather, he will be positioned in the location where the team can make the most plays.

-- MWE
   5. Mike Emeigh Posted: November 21, 2002 at 02:05 AM (#607347)
Phew! That's a lot of work.

Yes, it is. And I suspect that Brock Hanke's Law of Diminishing Returns will kick in here before much of it gets done.

It is possible to identify ways to improve any analytical method we have out there. I know that Tango and David Smyth and others over on FanHome have spent a *lot* of time doing just that with offensive methods, I know MGL has done the same with super-LWTS, and I know Charlie is always looking for ways to improve CAD and that Clay is always looking for improvements to the DFTs (which have had, by my count, at least three major changes since they were originally published in 1998). At some point, though, the extra improvements are likely to require a significant amount of effort and will likely make the method more complicated, and we have to decide whether the additional complexity adds enough value to make the effort worthwhile. If it isn't likely to add much value, perhaps it's best to acknowledge the shortcomings and move on.

-- MWE
   6. Mike Emeigh Posted: November 21, 2002 at 02:05 AM (#607349)
So Jeter's poor fielding stats that are primarily a function of total chances could possibly be more a reflection of Jeter vs. Brosius/Ventura than they are a reflection of Jeter vs., say, Garciaparra. Am I thinking this through properly?

I think so. That's part of the rationale (IMO) behind James's approach in Win Shares - which is why I find his railing against Nap Lajoie in the book for taking all of the discetionary putouts mildly amusing. I don't think it occurred to James that Lajoie might have had so many putouts *because* he was better at the things that get 2Bs putouts (judging popups, handling the ball on a DP pivot or on a steal attempt) than his SS were at those things.

-- MWE
   7. tangotiger Posted: November 21, 2002 at 02:05 AM (#607351)
Art: absolutely! Abso-freaking-lutely.

This is why I find it very surprising that Bill James did not do this. I mean, I would have a sabermetric orgasm if I had access to what Bill had, and get to spend all day doing it too.
   8. Mike Emeigh Posted: November 21, 2002 at 02:06 AM (#607357)
positioning is a huge part of competent defense, and if a fielder has to move to compensate for weakness, any resulting drop in UZR would appear to be a fair reflection of poor fielding ability and performance, just as any increase in UZR based on superior positioning would appear deserved.

I don't believe that Jeter is positioned differently because of weakness, except to the extent that the 3Bs are more likely than he is to convert balls in the hole into outs. He's positioned differently primarily because, based on the BIP distribution against the Yankees, it's in the team's best overall interest to shade him toward the middle. Jeter will convert fewer balls in the hole into outs not primarily because he's weak at doing it, but primarily because he's been moved further away from it.

We have to get away from looking at fielder performance in isolation, I think. If a ball gets by a fielder, and we think it's because he was out of position, the first question we need to ask is whether under the circumstances he SHOULD have been in position to field the ball.

-- MWE

-- MWE
   9. tangotiger Posted: November 22, 2002 at 02:06 AM (#607364)
Slammer: have you looked at the UZR results for JEter? PLUS 8, minus 1, minus 1. (In 2001, he was I think minus 17).

I'd like to see his 2002, but, where is the "consensus"? The consensus among the measures that have similar biases?

   10. Best Regards, President of Comfort, Esq. Posted: November 22, 2002 at 02:06 AM (#607365)
Slammer--

If all the different fielding metrics find that Jeter's a bad fielder in spite of their weaknesses, isn't that something of a consensus? Even if each one has its flaws, they're all finding the same thing.

But if they're all finding that Jeter is a bad fielder because of their flaws, then it is most definitely not a consensus. What Mike is trying to do is find a fielding statistic that doesn't have the flaws the other systems do, and see how that rates Jeter's defense.
   11. RP Posted: November 22, 2002 at 02:06 AM (#607366)
This is the same point I raised in an earlier article. If several *different* systems rate Jeter as a terrible fielder, is it more likely that each system is wrong, or that Jeter isn't very good? I think the latter is far more logical and likely. As far as the UZR stats, they show him to be, at best, a mediocre fielder. I don't necessarily see UZR as being all that inconsistent with the others.
   12. Mike Emeigh Posted: November 22, 2002 at 02:06 AM (#607368)
What Mike is trying to do is find a fielding statistic that doesn't have the flaws the other systems do, and see how that rates Jeter's defense.

Sorry to disappoint, but I haven't found one, and I do not believe that there is one (at present), although not for lack of effort. What I am trying to do, mostly, is to help people understand where we are and where we need to go to get past the flaws in the current systems.

-- MWE
   13. Charles Saeger Posted: November 22, 2002 at 02:06 AM (#607374)
Shredder, Joe M, others:

I don't think Mike would disagree here. In fact, Mike has given the same assessment of Jeter's fielding -- a low C-/high D+ shortstop.
   14. Chris Dial Posted: November 22, 2002 at 02:06 AM (#607375)
This should be helpful in understanding ZR, although it too needs to be updated. I still have the DP comment wrong.

Tippett is wrong on all three counts:
First, counting "all" the balls is a mistake in evaluatng fielders. ZR doesn't have the zone assignments perfect, but there are balls that simply can't be fielded with any sort of alignment that would actually take place in a MLB game. GBs down the lines are the responsibility of the 1B/3B, so he's mistaken there. And that only balls hit "more or less" at the fielder are all that's counted is absurd. We know the zones of responsibility are 30-40 feet wide. If that is "more or less at", then I won't disagree with Tippett on his point, but on his definition of "more or less".

Second: corrected.

Third: Tippett says, "At face value, you'd think this means that he covered more ground than the average second baseman." One would think that, if that were what ZR purports to define. It's not. Hits and errors are "plays not made". When we see that Alfonzo has a ZR higher than average, we aren't supposed to think, "He covered more ground." We are supposed to think he converted more balls hit into the 2B area into outs. And that's *not* a problem with ZR - that's a problem with a reader not understanding what ZR is.

Mike follows these "flaws" with a defeating statement:
"For these reasons, Zone Rating is not much of an improvement over the approaches based on analysis of traditional defensive statistics..."

Those aren't correct statements, so those aren't reasons, and ZR is significantly better. It is *not* perfect. It does have flaws, but those flaws are a function of STATS handling of the data, not the data itself. Mitchel Lichtman goes a *long* way towards correcting the problems with ZR.

The fact that the rankings are similar is not justification for lumping it with traditional-based stats - BA will rank players similar to PRO+ or RC/27, but that doesn't make BA a good descriptor of offense.

Bill James says in the Win Shares intro something like any stat that is right 70% of the time is a piece of crap (he actually likens it to a plane landing safely 70%, IIRC). Well, that's just what WS does. Now *that's* ironic.

Also, wrt UZR - weighting the plays by % of outs in an area does *not* weight them by difficulty. This aspect is an argument against static zones (which I support). Andruw Jones plays a clearly shallower CF than (some) other CFs. He catches a different set of balls. There is no way his regular ZR could be so average is he ran down everything. There are still 50 balls in his ZR zone he doesn't catch. Because we know he plays shallow, the balls he doesn't catch are likely to be over hs head. The balls he catches "out of zone" are likely to be balls in front of his zone, and not caught at a high rate by CFs, because they play deeper. So AJones gets +0.9 for catching a (would-be) single, and -0.5 for letting a double go over his head. You can build a good mark doing this. Another issue is pop-ups. Andruw Jones (according to Mike's PBP data) takes the discretionary pop-ups. That counts a lot in UZR and the "newer" ZR. Before anyone starts whining about my personal vendetta against Andruw, I just want the facts to speak.

Don Malcolm's data, while interesting, is pretty much throwaway info. It's V?r?s' Law in action. We are talking about 120 chances for the SS/2B, from what I can tell. I don't have a BBBA '98 (and I'd love to have one!), so maybe Don can chime in with some raw data.

For specific observations on Jeter:
I think he plays a step closer to home. That decreases his range side-to-side. He's below average, but not a complete truckload of suck.
   15. Charles Saeger Posted: November 22, 2002 at 02:06 AM (#607376)
Chris: Jones taking discretionary popups would count little in UZR. MGL/Mike E, correct me if I am wrong, but making extra of a play that has a 10% failure rate, those plays would only count 10% towards a fielder's UZR. Making extra line drives, however, would count more.
   16. Chris Dial Posted: November 22, 2002 at 02:06 AM (#607377)
Yes, Charlie, that could be correct, unless it is based on "turned into outs by other CFs", which I thought it was.

This just in: I've been wrong before.
   17. Mike Emeigh Posted: November 22, 2002 at 02:06 AM (#607378)
That's exactly the point I was trying to make. Yes, all the systems have shortcomings. But they don't all have the SAME shortcomings.

In fact, except for UZR, all the systems have essentially the same major shortcoming; they all penalize players for plays that they never had a chance to make.

-- MWE
   18. Mike Emeigh Posted: November 22, 2002 at 02:06 AM (#607379)
First, counting "all" the balls is a mistake in evaluatng fielders. ZR doesn't have the zone assignments perfect, but there are balls that simply can't be fielded with any sort of alignment that would actually take place in a MLB game.

It is my belief (as I argued earlier) that there aren't that many such balls than can't be fielded in *any* alignment - no more than 1% of the total number of balls in play, and that trying to identify that small fraction requires more effort than is justified for the likely benefit. There are certainly balls in play that can't be fielded in a *particular* alignment, but we don't record where the defenders were positioned when the ball was put into play, and until and unless we do we can't make an accurate determination on a particular BIP.

It's not. Hits and errors are "plays not made". When we see that Alfonzo has a ZR higher than average, we aren't supposed to think, "He covered more ground." We are supposed to think he converted more balls hit into the 2B area into outs. And that's *not* a problem with ZR - that's a problem with a reader not understanding what ZR is.

What most people tend to think when they see that player A has a higher ZR than player B is that player A is "getting to" a higher percentage of balls than player B. Tippett's three arguments against ZR (one of which is no longer appropriate) point out that the assumption isn't necessarily true. I'll concede that the problem isn't with the method, but with trying to make it a measure of an aspect of defensive ability that it wasn't intended to measure. That hasn't stopped people from trying to apply it in just that fashion.

It does have flaws, but those flaws are a function of STATS handling of the data, not the data itself.

Partially it's a function of STATS' handling of the data, and partially it's a function of the definition of the method (static zone assignments with no regard to BIP distribution, and de facto designation of some areas of the field as being unassigned).

Mitchel Lichtman goes a *long* way towards correcting the problems with ZR.

Absolutely. It still leaves the problem of accounting for variations in BIP distribution.

About Andruw:

There is no way his regular ZR could be so average is he ran down everything.

Making plays out-of-zone while allows balls in zone to drop will reduce your ZR, because of this feature of ZR:

"The player is credited with both an 'out' and a 'ball in zone' for balls caught outside his zone."

Let's say there are 200 balls hit in Andruw's zone and 50 loopers which he is in a position to field by playing shallow. The average CF will catch something like 90% of those balls, and none of the popups. Suppose Andruw were still able to catch 85% of the balls in the zone, but picks off 40 of the loopers by playing shallow. So he will actually have more *outs* made on those balls than the other fielders, but because the 40 popups he catches out of zone will be added to his opportunities, he'll have 240 opportunities instead of the 200 the other OFs get, and instead of a .900 ZR he'll post an .875 ZR, while trading 10 EBH for 40 outs. The Braves might like that trade.

Don Malcolm's data, while interesting, is pretty much throwaway info. It's V?r?s' Law in action. We are talking about 120 chances for the SS/2B, from what I can tell.

I KNOW it's not enough. I KNOW it's a small sample size. But it's all the hard data that we have, and the conclusion from the analysis is supported by inferences that can be drawn from the PBP data. Hitters get a higher percentage of hits on GB hit into areas where the fielders are less likely to be playing based on the *normal* ball distribution. LH hitters get a higher percentage of hits when they hit a GB into z56; RH hitters get a higher percentage of hits when they hit a GB into z6M. That particular pattern is consistent with the conclusion that the ability of a fielder to convert a BIP into an out drops sharply if he has to move for a ball. Don's observations give us a hypothesis that explains some of what we see in the data, and should be a focal point for additional analysis, not either dismissed as inconsequential or intended as a period, finis, end of the debate.

-- MWE
   19. Mike Emeigh Posted: November 22, 2002 at 02:06 AM (#607380)
For specific observations on Jeter: I think he plays a step closer to home. That decreases his range side-to-side. He's below average, but not a complete truckload of suck.

Without tipping my hand too much - otherwise you wouldn't want to read the rest of the series :) - I'll just say that I can't find any reasonable interpretation of the data that would make Jeter anything close to a GG-caliber fielder. My SABR32 conclusion, before I looked at the data in more detail, was "average at best".

-- MWE
   20. Chris Dial Posted: November 22, 2002 at 02:06 AM (#607382)
There is an alignment to catch any BIP. But they aren't used. That doesn't count, IMO. It's way more than 1% of balls.

About Andruw:
Those are good estimates, Mike. I actually think you have the numbers pretty close - actually, I may lean towards 50-60 extra outs based on his pitching staff GB/FB tendencies. Well, it's 400 outs and 440 for Andruw. The bigger issue is - how many of those extra catches are discretionary popups and flyballs? In a three-year sample, AJones caught 50 more pop-ups than average. That's 18 a season, so his 50 "extra plays" is down to 32 while giving up 10 XBH. And maybe that is a benefit (um, 10*1.1 - 32* 0.75 = 13 runs). But does that make him the greatest defensive CF ever? Hardly (and I know you don't claim that - some crackpot did). If they counted balls out of zone as "make-up" plays (as they should), Andruw would be nearly perfect - if the distribution wrt zones is as we are theorizing. Maybe he is better than I think - but that depends on the in zone/out of zone ratio.

Static zones *can* work because MLB teams play the same defensive alignments. SS all stand in the same spot (really really close to it).



   21. Chris Dial Posted: November 23, 2002 at 02:06 AM (#607384)
Tom,
well, only 30% of BIP are hits, and some of those (say 12% are in a fielder's zone. So that leaves 18% of balls that may or may not be "fieldable" but not in a player's zone. Figure that many of those hits fall into areas where out conversion is very low, but exists: the z56 and z34 and up the middle, and FBs to the wall and FBs in the gaps. Suddenly you are down to 5%. So sure maybe it is 1-5% of BIP are unfieldable, but that represents 6-18% of BIP not turned into outs usually. That *is* a large portion, and plenty of reason not to include the areas of the field as any player's responsiblity.

   22. MGL Posted: November 23, 2002 at 02:06 AM (#607385)
I'm working this weekend on (among other things) redoing my UZR calculations (one I revamp the progrtams it is nothing to run off any year for which I have PBP data). The only thing I am going to change for now is to use separate baselines for: 1) runner on first and no runner on second versus all other situations, for the first baseman only. IOW, when the first baseman holds the runner on versus when he doesn't; 2) DP situation versus no DP situation for the 2B and SS only; 3) MAYBE I'll use infield in as a separate category for the whole infield, although there is no designation in the PBP data for the infield playing in - I have to infer it from the inning and score. 4) Maybe I'll eliminate when the outfield plays in (9th or later, home team up, score tied, runner on third, less than 2 outs) from the data, but I'm not sure it's worth the trouble. 5) In addition to any or all of the above categories, there will be a separate baseline for the batter/pitcher R/L matchup. I think I'll use all 4 categories (RHP/RHB, RHP/LHB, LHP/RHB, and LHP/LHB), although I think that someone said that only the batter mattered - if that's the case and I can use only 2 categories, it will help with sample size (too many categories of anything, even if those categories are legitimately related to the results, causes larger and larger sample error in the results because of the large sample errors inherent in lots of small categories).

One more comment and I'm off to work. Should any PBP (or pseudo-PBP - well maybe ANY system) based system give more weight to errors than to "balls not fielded". For example, in ZR and in UZR, errors are treated exactly the same as "balls not fielded", which at first glance seems correct.

(BTW, I agree wholeheartedly with whomever disagreed with whomever "criticized" ZR for sometimes giving the impression that a player with low error totals and a good ZR has good RANGE. That criticism is silly for the reasons already posted. ZR makes no claim that it is a measure of "range" or of "good hands" or anything in between - nor does it have to. It is what it is, and all you have to do is read the one or two sentecnes that explains what it measures. If you don't and you "assume" that it measures range, then that is your problem. It is not a "problem" or wekaness of ZR. That is silly logic.)

Anyway, back to the "errors". When an error is recorded for a fielder, we are pretty darn sure that it is indeed a play that a fielder SHOULD have made but he didn't (yes, we all know that the official scorers sometimes make errors in judgment - that doesn't affect my argument here). When a SS does not make a play in his zone and gets docked the same as an error (in ZR) or when he doesn't make a play on a ball hit in a zone that an average SS makes a play on 60% of the time and he gets docked .6 of an error, we are not so sure that that should be the case. It is entirely possible that some of the balls that Jeter didn't get to in his zone last year, for example, were balls hiot at the extreme outer limits of his zone, whereas the balls that another SS didn't get to were hit in an easier area of the zone. In fact, look at it this way. AN error is really the same thing a s missed ball in a zone EXCEPT we know that the ball was hit pretty much at the fielder (more or less), so we don't worry about the possibility of the ball being in the outer limits of a player's zone. The more that I think about it, I really think that errors should be given more weight than missed balls in a zone, although I have no idea by how much. Actually, the MORE that I think about it, my argument is only relevant to ZR, not to UZR. In UZR, I treat errors separately. It doesn't matter what zone they are hit into. In fact, I treat them as if they werer hit into a zone that the particular fielder makes plays 100% of the time - which is the correct way to handle them. ZR, however - now I am sure of it - does not handle the errors correctly. They cannot be treated as mere missed balls (which I think they do, don't they?). Let's say the average ZR for a SS is .750 (the average SS fields 75% of the balls hit into some arbitrary zone or zones). For every missed ball, ZR assumes that the fielder should have caught it 75% of the time. If an error is treated the same as a missed ball, which I think it is, that assumption is wrong, of course. An error should have been fielded 100% of the time, by defintion. So a SS who fields 70 balls, misses 25 and makes 5 errors, will have a ZR of .700 - is that correct? A fielder who fields 70 balls, misses 10 and makes 20 errors also has a ZR of .700 - is that correct? That is horrible! Obviously the first fielder is much better! Let me see if I can figure out how to adjust the ZR to account for errors as opposed to missed balls. It's a little tricky.....

I'm drawing a blank here, but I'll take a stab. A missed ball costs the team around .75 outs (isn't fielded versus should be fielded 75% of the time), whereas an error costs the team around 1 out (isn't fielded but should be fielded 100% of the time), so an error is 133% more costly than a missed ball. I think then that an error should be worth 1.33 missed balls. So for our first player he has 25 misses and 5 errors or the equivalent of 25 + 5 x 1.33 misses or 31.67 misses and 70 plays for a ZR of 70/101.67 or .689! The second player has 10 misses plus 20 errors, which is the equivalent of 10 + 20 x 1.33 or 36.6 misses, for a ZR of 70/106.6 or .657! Of course, using this new system the average SS ZR would have to be calculated the same way - converting the average SS errors into misses (by multiplying by some number like the 1.33), adding this to his misses and then dividing his total plays by his total plays plus "misses". I out the last "misses" in quotes obviouslyu becuase they are not the real number of misses by an average fielder - they are the real number of misses plus the errors times some coefficient.

Could I have found a major flaw in ZR, which should be corrected by STATS or whoever calculates ZR, and one that should have been obvious a long time ago (sometimes we overllok the obvious)?
   23. Chris Dial Posted: November 23, 2002 at 02:06 AM (#607387)
MGL,
I believe the error will be offset by the base advancement. An error is like an infield single - worth slightly less than an actual single. I don't believe it should be counted like you suggest.
   24. Mike Emeigh Posted: November 23, 2002 at 02:06 AM (#607388)
Tom, well, only 30% of BIP are hits, and some of those (say 12% are in a fielder's zone. So that leaves 18% of balls that may or may not be "fieldable" but not in a player's zone. Figure that many of those hits fall into areas where out conversion is very low, but exists: the z56 and z34 and up the middle, and FBs to the wall and FBs in the gaps. Suddenly you are down to 5%. So sure maybe it is 1-5% of BIP are unfieldable, but that represents 6-18% of BIP not turned into outs usually. That *is* a large portion, and plenty of reason not to include the areas of the field as any player's responsiblity.

That would be true if defensive alignments didn't change - but they do change. They change based on the game situation (infield up or back or at DP depth; 3Bs guarding the lines in the late innings, 1B holding the runner on and charging in a bunt situation), the hitter (overshifts against the Bondses and McGwires of the world; outfielders deep or shallow and swung left or right), the ballpark (you can't play deep in LF at Fenway or Minute Maid or in RF at OP@CY, and you need to play deeper at Coors Field), and the pitcher (does this guy work inside or outside? up in the zone or down in the zone?).

My argument isn't that there are no uncatchable balls, but that:

-- the determination of whether a ball is catchable or not is dependent on the defensive alignment, and

-- the defensive alignment is variable, based on the combination of game situation, hitter, pitcher, and ballpark.

Therefore, you cannot make an assumption in advance that any particular ball remaining in the field of play will always be uncatchable. For almost every ball put into play, there is some combination of game situation/hitter/pitcher/ballpark that dictates a defensive alignment in which the ball can be turned into an out, and unless you are certain that the ball was not put into play under those circumstances, you cannot justify excluding it from your analysis.

-- MWE
   25. MGL Posted: November 23, 2002 at 02:06 AM (#607389)
Chris, I'm not talking at all about the run "value" of the error versus that of the missed ball, although if it were true that the run value of the error were 2/3 or so that of a hit (missed ball), then yes, you could make an argument for treating them exactly the same, even in a ZR.

Of course that is IF that were true....

The average value of a GB hit (1999-2001) is: .503

The average value of GB error (ROE) is: .509

Chris, you should have known (guessed) that it would be close! Shame on you! An error by an infielder is sometimes like an infield single and sometimes like a double or worse (when the IF throws the ball away), whereas a GB hit is usually a single and occasionally a double or triple...

BTW, the value of all ROE's (including OF errors and IF errors other than on ground balls), is .522.

Interestingly, while the value of an out for a batter was -.286, not including errors (IOW treating a ROE as a hit), it is -.303.

Here are some other values, for reference:

Out: -.290
BIP out: -.286
K: -.304
Out no DP: -.274
GDP out: -.884
GB out (no DP), runner on 1st, less than 2 outs, no bunt: -.299
FB out: -.282
GB out -.285
LD out: -.310
IBB: .192 (obviously not considering individual batters)
BB: .33
s: .48
d: .78
t: 1.05
hr: 1.39
sb Including catcher errors, I think): .17
cs: -.47
sh, runner on 1st: -.14
sh, runner on 2nd: -.08
sh, runners on 1st and 2nd: -.05
suicide squeeze: .04
   26. Robert Dudek Posted: November 24, 2002 at 02:06 AM (#607392)
Hi Mike,

I'd like to join the chorus and thank you for making these articles available to us.

I've taken you GBIP numbers and looked at the 3B and SS zones. If I am correctly interpreting the numbers, the balls hit into zones ranging from the 3rd base line to the middle of the diamond are as follows:

3B line/direct: Yankees 134, League 2330
3B-SS hole: Yankees 416, League 6628
SS direct: Yankees 333, League 5426
SS middle: Yankees 377, League 6907

If we add up all these balls in play, we get: Yankees 1260, League 21291

If we take the League distribution and apply it to the 1260 balls in play against the Yankees, we get something like this:

3B line/direct: 137.9 (Yankees' net -3.9)
3B-SS hole: 392.2 (Yankees' net +23.8)
SS direct: 321.1 (Yankees' net +11.9)
SS middle: 408.8 (Yankees' net -31.8)

What this suggests to me is that, in comparison to the League distribution of groundballs towards the left side of the diamond, the Yankees receive many more balls in the SS-3B hole and (to a lesser extent) directly at the SS. The greatest difference is the relative scarcity of balls hit towards the shortstop side of 2B.

Given this, wouldn't it make sense to shade BOTH the 3B and SS towards the hole a bit (compared with "normal" League positioning?



   27. tangotiger Posted: November 24, 2002 at 02:06 AM (#607393)
MGL said although I think that someone said that only the batter mattered . Yes, I said it after I saw one of your research results!

As for the error thing you mentioned: let me think about it. This is interesting.
   28. MGL Posted: November 24, 2002 at 02:06 AM (#607394)
I assume that the reason that the Yankees saw (I don't know what year(s) Robert is referring to) more balls hit in the SS hole and directly at the the SS, as opposed to "up the middle" is because they had at least 2 LHSP in Pettitte and Wells (hence faced more RHB) AND that both of these pitchers tend to pitch inside to RHB's.

I would guess that the fewer balls hit down the third base line is a function of sample size (just fluctuation). I think that if you face more RHB, you will shift all your GB's to the left side od the IF. I don't think that it is within a pitcher's control to shift only certain locations to one side or the other, despite the fact that it looks like the Yankees saw GB's "bunched" in the hole and at the SS.

So yes, when a LHP is on the mound, especially Pettitte and Wells, Jeter should position himself more towards the hole, which I'm sure he does, but the 3rd baseman should position himself more towards the line, despite the fact that the Yankees apparently saw fewer than average balls hit down the line. Basically, an infield should shift one way or the other en masse, depending upon how much of ground ball pull hitter the batter is versus the particular pitcher on the mound (all batters pull more than 50% of their ground balls). Any unusual infield alignment you see, like the "bunching up" of adjacent fielders, is probably sub-optimal, and probably based on a small sample of a hitter's scouitng report (scatter diagram of his GB's).

The moral of the infield positioning story is that any good fielding metric must adjust (if it isn't already built into the stat) for an "assumed" shift in positioning, based upon the handedness of the batter/pitcher matchups. UZR should do this, which it eventually will, and the "context-adjusted" fielding stats should do this as well. Even ZR should change each fielder's zones of responsibility (if they are small enough) to account for this. Interestingly, positioning is already built in to the traditional fielding stats like FR and RF...
   29. tangotiger Posted: November 24, 2002 at 02:06 AM (#607397)
MGL, I thought about your error / ZR thing. I don't think it'll fly.

Now, your point is that since we "know" that those 10 errors occurred in the "100% out-conversion zones", then we should treat them somewhat differently.

Take for example the league average 70 outs made, 20 balls missed, and 10 errors. Let's assume that Jeter makes 70 outs, 30 balls missed, and 0 errors.

The "normal" distribution is to have 60% balls in the "99% catchable" zones, and 40% balls in the "tough zones" (for example).

So, our league average SS made 50 outs and 10 errors (and 0 misses) in the catchable zone. He was 20 outs and 20 missed in the "tough" zones.

Jeter made 60 outs (and 0 errors and misses) in the catchable zone. He made 10 outs and 30 missed in the "tough" zones.

So, Jeter made 10 more outs in the catchable zones, and 10 less in the tough zones. It doesn't matter that you have balls that are flagged as errors or not. They are effectively hits, and they can't be used as you suggested.

I think.

   30. MGL Posted: November 25, 2002 at 02:06 AM (#607398)
Tango,

Your logic is wrong and I'll explain why (maybe) later. I CAN explain how my logic (that errors have to be treated differently than missed balls in ZR) HAS TO BE correct.

One reason we don't like ZR is because it treates all balls equally - i.e, it does not differentiate between hard or easy to catch balls. As with any measurement that is "reliable" but not "accurate" (such as BA as comparted to OPS), it does not pick up the "fine distinctions" that the more accurate metric does pick up. For example, ZR is not able to pick up the distinction between player A who has a ZR of .750, but who had more balls hit into an easy zone(s) and fewer balls hit into a tough zone(s), as compared to player B, who may also have a ZR of .750.

Now let's think about ZR and UZR with respect to errors. Suppose that we have only 2 zones per position - an easy zone and a hard zone. No,let's say 3 zones - an easy zone, more or less right at a fielder, and TWO hard zones, one to each side of the fielder. That is easier to conceptualize.

Let's say both players have the same ZR. Without knowing the distribution of balls hit into the easy versus the hard zones, we think that both players are of equal defensive value. That's fine. Now let's say that even though both players have the same ZR, we are given some more information. Let's say that we are told that player A missed 10 balls in the easy zone and that player B did not. As you correctly stated in your post: 1) we can assume that both players had an equal number of balls hit into each zone; 2)therefore, player A must have fielded 10 more balls in the hard zones than did player B, in order for their overall ZR's to be the same, and; 3) player B must have fielded 10 more balls in the easy zone than did player A.

So the question is, which is more valuable - fielding 10 more balls in the hard zone or 10 more balls in the easy zone. Your first instinct might be to say that it is the former, because fielding a ball in a hard zone requires more effort and skill than fielding a ball in an easy zone - which is true. But the kicker is that if you don't field a ball in an easy zone (an error), it is more costly than if you don't field a ball in a hard zone, since that ball probably would not have been fielded anyway! In any case, if you do a UZR calculation on our 2 players, you will come up with a better number for player B (the one without the errors), since we "dock" player A around 1 out for each of those extra 10 balls he misses in the easy zone, as compared to player B, and we only dock player B .5 outs (or whatever the number is) for each of those 10 extra balls he misses in the hard zone.

The whole thing boils down to, if we have a ZR, AND we have a little UZR information, in the form of errors (missed balls in an easy zone), we might as well make use of that extra information. Using a UZR methodology, we increase the value of player B (fewer errors) relative to player A (more errors), so the only way to incorporate that change into a ZR is to weight an error more than a missed ball.

If you diagree or don't understnad the above argument, here's another way to look at it which should convince you that errors have to be woth "more" than a missed ball, ZR being equal of course. It is often helpful to exaggerate a situation to the poin tof being ridiculous in order to understnad what is going on. Analysts do it all the time, when a situation, like this, is hard to fugure out.

Let's say that a player has 1 assists and 99 errors. That means that 99 balls were hit in his vicinity, he booted them all, and that 1 ball was hit somewhere in his zone of responsibility (let's assume that that ball was not hit outside his zone). His ZR is .01. We don't know where that 1 ball that was caught was hit, but it doesn't really matter (all caught balls tned to be hit into easy zones, and all missed balls tend to be hit into hard zones).

Let's say that player B also has a zone rating of .01, but he has no errors.

Which player is the better fielder? Whom would you rather have on your team?

We know that player A can't catch a ball to save his life, even if it is hit right at him. In fact, player A, if you didn't know it, might be a 3 or 4 year (Dusty Baker's son?) old playing in a major league game (seriously).

Player B, on the other hand, had no easy balls hit at him, other than perhaps the one he caught. We are pretty sure that player B has no range, bu tothe than that, we don't know that much about him. In fact, it would be a pretty good assumption that most of those balls hit in his zone were hard to field. It might even be that ALL of those balls, other than the 1 he caught, were impossible to field. Not so with player A! IOW, a player's errors tells us something about the distribution of balls hit into the various zones (the more errors a player has, the more likely it is that he had more balls hit into he easy zone)!

Here is another example that is analogous to BA. Suppose player A had a BA of .300 and so does player B. Who is the better player? Obviously, we don't know. What if I tell you that player A has at least 10 HR's but we don't know how many HR's player B has. Who has the most HR's? Who is the better player? The answer is player A, even though the average number of HR's might be 20. I think you know why. Since initially, player A and player B each have the same chance of having 0,1,2,3,.....60+ HR's, once we say that player A has at least 10 HR's, we eliminate from his "possible HR's", 0-9, therefore his average posssible HR number is higher than player B. Actually, a better analogy is "warning track outs" rather than HR's.

Same thing for ZR (BA) and errors (HR's). If two players have the same ZR, and A has 10 errors B none, we now know that player A has at least 10 balls hit into his easy zone! So who is likely to have the greatest number of balls hit into the easy zone, player A or player B? Player A, of course! If 2 players have the same ZR, but player A is more likely to have more balls hit into the easy zone, who is the better fielder? Player B!

Now, if you want ZR to measure "performance" and not "ability" then yes, a missed ball is the same as an error, of course. But, as I always say when it comes to performance versus ability, who the heck would care about ZR as a measure of performance? When we look at a player's ZR and comapre it to another player, 9 time sout of 10 (or more), we are trying to anser the question, "Who is (or even was) the better fielder?" The real importance of adjusting ZR for errors is that if we don't, ZR correlations from year to year will not be as high as if we adjust for errors, and people will tend to doubt them as indicators of fielding talent OR performance...

Bottom line, if you want ZR to represent ability, and thus, to give you the best estimate of ZR for subsequent years AND to enable you to fairly and accurately comapre one player to another, you should adjust for errors. If you want to ZR to represent value (how many balls hit in a designated area a fielder fielded), then there is no need to adjust for errors.

It wouyld be the same argument for batters and BA in the following scenario:

Batter's A and B had the same BA in the same number of AB's - say, .250. I then tell you that batter A had 20 screaming line drives caught by an IF'er and batter B had only 3. WHo is the better player? Batter B! Who is more likely to have the higher BA next year? Batter B? Should we adjust every player's BA for his line drives caught if we want to compare one player to another to see who is better or who is likely to have the higher or lower BA in a subsequent year? Sure! Do we have to? No!
   31. MGL Posted: November 25, 2002 at 02:06 AM (#607399)
Tango, I made the same mistake in paragraph 4 of my post that you made in your post (that was your mistake in logic). The likely distribution of balls, as far as whether they were hit into hard or easy zones, is NOT the same for player A and player B. Once we know that player A had more errors than player B, it is likely that player A had more balls hit in the easy zone - but not 10 more!

Your mistake was assuming that both Jeter and the average SS had the same number of balls (60) hit into the easy zone. Once we know that Jeter had 0 errors, and the average fielder has 10, it automatically means that he had fewer balls hit into the easy zone, so his distribution does NOT look like 60 outs in the catchable zone and 10 outs and 30 missed balls in the tough zone. It is more like 55 or 57 outs in the easy zone, 13 or 15 outs in the tough zone, and still 30 missed balls in the tough zone.

Same logic as if I told you that player A has 20 HR's in 500 BIP, and the average payer has 15, is player A likely to have more or less FB than the average player? More, of course. (A HR is a fly ball, therefore we know that player A has at least 20 FB's. All we know about the average player is that he has 15 HR's, so he has at least 15 FB's. Who has more FB's on the average, a player with at least 20 or a player with at least 15?)
   32. tangotiger Posted: November 25, 2002 at 02:06 AM (#607405)
MGL's last post essentially summarizes his position. That because errors are done in the catchable zones, then the more errors in that zone, the more likely that the player had catchable balls overall.

The more HR a pitcher gives up, the more likely he has more flyball outs. The more SB a runner has, the more likely he has alot of singles and walks, etc, etc.

But, to what is the degree of impact of this as it applies to errors? Assuming that a SS makes between 0 and 50 errors a year, and he has about 700 plays to make (with 500 outs, 25 errors, and 175 misses), let's try to figure that out.

Our SS has 25 "known catchables" and 675 "unknown" plays. Let's assume that of all unknown plays, two-thirds are catchable (450), and one-third (225) are tough.

This breaks down to catchables: 450 outs, 0 misses, 25 errors (475 total plays)
tough: 50 outs, 175 misses, 0 errors (225 total plays).

How about for Ozzie? Well, this shortstop has zero "known catchables" and 700 "unknown" plays. Assuming the two-thirds breakdown on unknown plays, that gives Ozzie 467 catchables, and 233 tough.

Let's assume that Ozzie went 500 out, 0 errors, 200 misses. So, the break down is now
catchables: 467 outs, 0 misses, 0 errors (467 total plays)
tough: 33 outs, 200 misses, 0 errors (233 total plays)

So, here are the out-conversion rates for Ozzie and the league in each zone
catchables: 100%, 94.7% (+5.3% for Ozzie)
tough: 14.2%, 22.2% (-8% for Ozzie)

If we give Ozzie the distribution of balls our league SS got, that would give him 100% x 475 + 14.2% x 225 = 507 outs, compared to the league average of 500 outs, giving Ozzie 7 more outs per 700 balls in play. And this is if he had 25 less errors than the league average. And this is assuming that MGL is right about the breakdown of unknownables.

I'd like to see some data on the relationship between errors and ball distribution.
   33. Chris Dial Posted: November 25, 2002 at 02:06 AM (#607406)
Yes, errors may be very slightly more costly than missed balls.

However, your logic is flawed with the giant assumption that errors are charged on "easy" plays. The ZR zones should *all* be easy plays. They are defined by the areas of the field where >50% of plays are made (according to STATS). If you want to break down the data in terms of chances for Derek Jeter based on the actual ZR zones (as defined in the article I attached), then you can illustrate that Jeter gets more chances that are 12 feet away from him rather than Omar Vizquel who is always fielding balls hit right at him. Of course, that brings Mike's point in play: were the fielders standing in the same spot?

Errors have less base advancement. You also illustrate by GB error data for "run value". The difference is 1%. So if you want to count the error 1.01 "misses" instead of 1 "miss", you are wasting a lot of time for no gain in accuracy, simply due to the difference in official scorers.

As for "ability" vs. "performance", this only holds if *errors* are constant. If a fielder makes 15 errors in season 1 and 10 in season 2, is that due to his "ability", the official scorer, or a function of the slight difference in chance distribution.
   34. Mike Emeigh Posted: November 25, 2002 at 02:06 AM (#607407)
I think that if you face more RHB, you will shift all your GB's to the left side of the IF.

That was specifically *not* what the Yankees saw, though. The Yankees saw more RHB than the norm, because of the LHSP that they had - but the overall distribution in the infield skewed in the other direction. The fact that the skew *around SS* was toward the hole doesn't change the fact that there were more chances than should be expected on the right side of the infield. (See Table 9 in Part 5).

It is this odd ball distribution that is the key to Jeter's low rating in most defensive analysis systems. The Yankees had a BIP distribution that suggests a right-side skew, when most defensive methods would predict a left-side skew. In addition, because Jeter's specific skew at SS was toward the hole, he faced a higher percentage of difficult plays (made more difficult by the likelihood that he was shading in the other direction) relative to the average SS. He didn't get a whole lot of balls hit directly to him.

Should Jeter have been shading toward the hole more, or playing a normal defensive position? Perhaps (almost certainly in 2000 when Brosius had apparently lost a step), but that would have left an awfully big hole up the middle, since the skew on the 2B side of the diamond was definitely toward the hole. Jeter doesn't appear to have great lateral movement, as Chris suggested earlier (it shows up pretty clearly in the DP-depth situations, as you will see in Part 7) and I think they'd have been taking a pretty big risk shading Jeter that way, especially when Brosius was in his prime and covering a decent-sized chunk of the hole.

-- MWE
   35. MGL Posted: November 25, 2002 at 02:06 AM (#607409)
Tango, no, no, no, no ,no! (As Ben Kingsley said in "Sexy Beast".) You keep making the same mistake!

If an average SS makes 25 errors in 700 chances and 2/3 of the remaining 675 are in a "catchable zone" and 1/3 are in an "uncatchable zone", then....

If a SS, like Ozzie, has no errors in 700 chances, his breakdown will NOT be 2/3 catchable and 1/3 uncatchable! The unknowns will NOT have the same catchable/uncatchable breakdown! Ozzie will have like 70% catchable and 30% uncatchable!

Do a sim and you will see how that works!
   36. tangotiger Posted: November 25, 2002 at 02:06 AM (#607410)
Actually, I am not making the same mistake, but a *different* mistake. The first time, I assumed that all players received the same distribution. This time I assumed that all players received the same distribution for the "unknowables", with the effect that the player with more errors will end up with more catchables (i.e., 67% of all unknownables + 100% of all errors).

But, what you are saying is that the player with more errors will receive a different percentage of the unknowables as well (something like a player with 0 errors will get 65% of all unknownables, a guy with 10 errors will get 68% of all unknownables, etc, etc). Am I getting you right?
   37. MGL Posted: November 25, 2002 at 02:06 AM (#607411)
Tango, yup! That is correct. And yes, it was 2 different errors. It is actually a selective sampling bias, but very subtle. I am going to write a little sim to show you how it works...
   38. Mike Emeigh Posted: November 25, 2002 at 02:06 AM (#607412)
Jeter doesn't appear to have great lateral movement, as Chris suggested earlier (it shows up pretty clearly in the DP-depth situations, as you will see in Part 7)

Actually, I thought it did, until I broke the data down by play type and saw that (like everything else) it wasn't that clear at all. So scratch that last comment :)

I'm trying to get Part 7 finished tonight - for a variety of reasons I wasn't able to do much work with it this weekend. Hopefully we can get it on the site by Wednesday at the latest, to give you all something to chew on besides your Thanksgiving turkey.

-- MWE
   39. MGL Posted: November 25, 2002 at 02:06 AM (#607413)
Tango, don't feel bad. This was not an easy one. Sometimes I have to set up a sim to confirm what I think is correct. Here is the sim:

Each fielder has 600 chances. I ran 100,000 sims.

Each fielder has 2 zones - a hard and an easy zone. In the easy zone, eac h fielder fields 100% of the balls. In the hard zone, each fielder fields 25% of the balls.

The program randomly "hits" ground balls such that 60% go into the easy zone and 40% go into the hard zone.

So each fielder has a true ZR of .700 (out of 100 balls, he fields all 60 in the easy zone and 25% of the 40 in the hard zone, for a total of 70 balls).

Then I had each fielder make an error on 10% of the balls in the easy zone. This gives each fielder 36 errors per 600 chances, and reduces their ZR to .640.

Let's see what the average percentages of balls hit in the easy and hard zones for players with various error totals after 600 chances....

Players with less than 36 errors had an average ZR of .645 and an average number of errors of 31.1 This is to be expected.

These players however, had 59.8% of all balls hit into the easy zone and 40.2% hit into the hard zone!

If we look at all players with error totals less than 25, we get:

Average ZR of .647, average # errors of 22.3, and 58.4% balls in the easy zone and 41.6 in the hard zone. (There were only 18 such players out of 1000, BTW.

On the flip side...

Those players who had error totals greater than 36 (430 players) had a ZR of .634, 41.3 errors, and 60.3% in easy zone, and 39.7 in hard zone.

For greater than 45 errors, we get .628 ZR, 48.7 errors, 60.9% easy and 39.1% hard.

You can do the math to calculate the hard/easy percentages for "non-error" balls.

You can see what is going on!

The next sim I will run (after I get back from class tonight) is the same as above, but when each player has a different true error rate...
   40. Chris Dial Posted: November 25, 2002 at 02:07 AM (#607414)
David, errors are scored by "ordinary effort" (as I'm sure you know). The entire SS zone is considered ordinary effort. It will depend on how sharp a ball is hit and whether the guy had to dive, but the zone size is set to intentionally be balls that a fielder would get an error on if he botched the plays.

   41. tangotiger Posted: November 25, 2002 at 02:07 AM (#607415)
MGL, very interesting!

Breaking your numbers down then:
...an average number of errors of 31.1. These players however, had 59.8% of all balls hit into the easy zone and 40.2% hit into the hard zone!

So, out of 700 balls, we have 31.1 "known easy" and 668.9 "unknownables". We know that 59.8% of the 700 were easy, or 418.6. Removing the 31.1 errors gives us 387.5 easy out of the 668.9 unknownables or a rate of 57.9%.

...average # errors of 22.3, and 58.4% balls in the easy zone and 41.6 in the hard zone. This works out to 57.0%. (Only 18 players in the sample.)

...41.3 errors, and 60.3% in easy zone, and 39.7 in hard zone. This is 57.8%.

...48.7 errors, 60.9% easy and 39.1% hard. This is 58.0%.

Seems to me that the rate for the "unknownables" *IS* the same, at 58.0%, or so, regardless of error rate. However, your point is also made. Let's call it a draw!

   42. MGL Posted: November 26, 2002 at 02:07 AM (#607416)
Tango, interesting...

I'll redo the calcs with larger sample sizes. You might be right that the percentages for all the unknowables is always exactly the same...
   43. MGL Posted: November 26, 2002 at 02:07 AM (#607417)
Here are the numbers for low and high # of errors. I ran 1,000,000 players with 600 chances each.

Less than 30 errors:

132,981 players. 26.8 errors. .646 ZR. .594 easy per chance. .575 easy per "unknowable" (non-error) chance. 2 SD = .003.

More than 42 errors:

135,805 players. 45.8 errors. .629 ZR. .607 easy per chance. .574 easy per "unknowable" (non-error) chance. 2 SD = .003.

It's obviously real close, but I honestly don't know, and we can't tell from the results of the sims, whether the easy/hard ratio for non-error balls is constant. I could easily have been wrong. Even if it is NOT mathematically constant, it is real close to being constant.

There is obviously a theoretical answer to the question whether of that ratio is constant, no matter how many errors. I just don't know how to figure it or conceptualize it.
   44. MGL Posted: November 26, 2002 at 02:07 AM (#607418)
I made a mistake in the standard deviation of "balls hit in easy zone per non-error balls" in my last post. I calculated it on 135,000 or so chances rather than players. With 135,000 players (i.e., player seasons), that is 135,000 players times 600 chances per player or 81 million chances. That yields a standard deviation (based on a binomial formula where p=.575 and q=.425) of .00005 and not .0015 as I indicated perviously. Given that, the .575 and .574 are statistically significant differences, which implies that the "easy ball per non-error chance" is very close for all error rates, but NOT exactly constant.

Anyway, back to the question of whether an error is worth "more" (in terms of ability) than a missed ball (I am 99% sure that it is), and if yes, by how much, I am running some more sims. This time I am looking at 30 fielders, with each one having a unique true ZR, based upon a unique true fielding % in the hard zone and a unique true error rate (in the easy zone). The way I assign each player with a unique true ZR is the following:

Once again, for all players, 60% of all chances are hit into the easy zone and 40% in the hard zone.

Player #1 fields 20% of all balls hit into the hard zone. Player #2, 20.25%, player #3, 20.50%, etc. IOW, I add an extra .25% to each player's true ZR. For example, player #30 has a true ZR of 27.50%. Again, all balls, other than an error, in the easy zone are fielded 100% of the time by all 30 players. However, each player also has a different error rate. Player #1 has an error rate of 5%, player #2, 5.25%, etc. Just like with the fielding % in the hard zone (20%, 20.25%, etc.), I add .25% to each player. For example, the 30th player has an error rate of 12.5%. Thus, the average % of balls fielded in the hard zone among all 30 players is .23875, the average error rate is .08875, and the average ZR among all 30 players is .64225.

I ran 10,000 seasons for all 30 players. A season is 600 chances.

I looked at all players who had a sample ZR for one season of between .642 and .644 (around average). Of those I looked at those who had high error totals and those who had low error totals. Remember that in calculating ZR, I am using the traditional definition - an error is the same as a missed ball.

Players (remember these are only players with one-year sample ZR's around average) with error totals ABOVE 36 (9619 such players) had an average sample ZR of .6416. Their average true ZR (ability) was .6389.

Players (remember these are only players with one-year sample ZR's around average) with error totals BELOW 36 (18932 such players) had an average sample ZR of .6417. Their average true ZR (ability) was .6444.

So the high error group's sample ZR overstated their true ZR and the low error group's sample ZR understated their true ZR.

Let's see what happens if I count an error as 1.33 missed balls (I add .33 chances for every error over 36 and subtract .33 chances for every error below 36....

The high error group now has a sample ZR (weighting an error more than a missed ball) of .6394 and a true ZR of .6389.

The low error group now has a sample ZR of .6451 and a true ZR of .6444.

Weighting the errors in a ZR seems to yield a better estimate of a player's true ZR (ability).

Two standard deviations for the above numbers is around .0003 for the low error groups (around 19,000 player seasons) and .0004 for the high error groups (around 10,000 player seasons).

The reason I only looked at players who had sample ZR's of around average (.642-.644) was: 1) I needed both the high and low error groups to have the same ZR; 2) I wanted to avoid any regression issues.

I know this last paragraph is hard to understand, but it's not that important.
   45. Mike Emeigh Posted: November 26, 2002 at 02:07 AM (#607420)
Dick Cramer sent me an E-mail in which he gave me a corrected history of STATS - thanks, Dick. STATS was founded in 1981, with Dick functioning as a part-time Chief Technical Officer; all of the fielding data recording formalisms went into use immediately thereafter. The parent company of STATS folded in 1984-1985, and Dick reorganized the company, bringing Bill James and John Dewan into STATS (John was, as you may recall, running Project Scoresheet at the time). In 1988 came the Great Divide: Dewan was removed from the board of Project Scoresheet, and James followed John over to full-time involvement with STATS, while Project Scoresheet evolved into the Baseball Workshop.

-- MWE

You must be Registered and Logged In to post comments.

 

All GBIP BIP PM %PM
Yankees 298 117 39.3%
AL 4792 2075 43.3%
       

 

<< Back to main

BBTF Partner

Support BBTF

donate

Thanks to
Tuque
for his generous support.

Bookmarks

You must be logged in to view your Bookmarks.

Syndicate

Page rendered in 1.1254 seconds
66 querie(s) executed