Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Primate Studies > Discussion
Primate Studies
— Where BTF's Members Investigate the Grand Old Game

Sunday, November 03, 2002

And the Beat Goes On: Derek Jeter and the State of Fielding Analysis in Sabermetrics

In the initial offering in his survey of defensive analysis, Mike charts the terrain to be covered.

Derek Jeter gets his share of praise for his defensive skills. In 2000,
American League managers rated him second among AL shortstops defensively,
behind only Omar Vizquel. In the 2001 AL Divisional Series, he made spectacular
plays in back-to-back games, first making a backhand flip of an off-line throw
to nail Jeremy Giambi at the plate, then tumbling head over heels into the
stands to grab a foul ball.

Derek Jeter also takes his share of criticism for his defensive performance.
Year in and year out, he makes fewer plays than any other AL shortstop. He ranks
at the bottom of the list in most well-known statistical measurement systems for
fielders.

href="http://espn.go.com/mlb/columns/neyer_rob/1415695.html">Rob
Neyer

notes that Jeter doesn’t even match the level of
performance turned in by his backups, while

href="http://www.baseballprospectus.com/news/20000802huckabay.html">

size=2>

Gary Huckabay

of Baseball Prospectus wrote this
in 2000:

“Defensively, Derek Jeter is positively horrible. He takes forever to release
the ball, has no range, his footwork was best described by Joel Grey in Remo
Williams: The Adventure Begins
and he will join the ranks of the defensively
horrific who win Gold Gloves because of their charisma, the occasional highlight
reel play, exposure and their offensive production.”

Sentiments similar to those above are expressed on internet mailing lists, in
newsgroups, here on Baseball Primer, and on other Web sites. Sabermetric
analysts are nearly unanimous on this subject - Derek Jeter is a horrible
defensive player. Almost every fielding analysis method out there ranks Jeter as
a below-average defensive shortstop, and most rank Jeter at the very bottom of
the list. But after evaluating the play-by-play record, I think those rankings
say more about the problems with existing statistical methods for evaluating
fielding than they say about Jeter’s fielding skill.

This series of articles expands on a presentation that I made at the 2002
SABR convention in Boston. In this initial article, I will lay out the series of
“first principles” that I believe should be an essential part of any defensive
analyst’s tool kit. In future articles, I will evaluate existing fielding
methods using Derek Jeter’s 1998-2000 season performance as an example, and
provide an assessment of each of those methods drawing on data from the
play-by-play data I have licensed from Gary Gillette and Pete Palmer for those
seasons. I will also look at the play-by-play data to assess Jeter’s fielding in
more detail.

I am not here to evangelize for (or against) a specific approach to fielding
analysis. I want to illuminate the problems inherent in current methods for
evaluating fielding, and to help readers understand that no single approach,
whether it be one of the zone-based systems or Win Shares or Clay Davenport’s
methods or Context-Adjusted Defense or Mike Gimbel’s system or Tom Tippett’s,
can be considered the definitive method for evaluating fielding. Most of the
authors of the aforementioned systems know their limitations, and make it
clear the extent to which those systems are limited. The rest of us need to have
the same awareness.

First Principles of Fielding Analysis

In evaluating a system for analyzing fielding performance, I make use of the
following “first principles.” I realize that some of these are not universally
accepted among analysts; nonetheless, after spending countless hours evaluating
the information available in the play-by-play data, it is my belief that a
fielding analysis system needs to be based on these principles if it hopes to
capture fielder skills accurately - specifically differences in fielder
positioning, which is in my opinion a little-realized aspect of fielder skill
that is not commonly addressed by analysts.


  1. There is no such thing as a “routine out” or (except for balls that leave
      the field of play) an “uncatchable ball.”
    Fielding skill includes
      positioning, quickness in reacting to the ball when hit, and hands; the first
      two of these combine to form what we commonly call “range.” If a fielder is
      positioned correctly, the ball is likely to be hit right to him and it looks
      like a routine out. If a fielder reacts to the ball quickly, he can usually
      get to a position right in front of the ball when it gets to him and again it
      looks like a routine out. But if the fielder doesn’t position himself properly
      or is slow to react when the ball is put into play, suddenly the play doesn’t
      look quite as much like a routine play any more. Even if he does react
      properly, if he mishandles the ball the routine out will likely go away.
      Similarly, an uncatchable ball for one fielder is often catchable for another
      fielder who is positioned a bit differently, or who reacts to the ball more
      quickly.


  2. When a ball that remains in the field of play is not converted into an
      out, the fielders should be held responsible for that failure, whether the
      result is called a hit or an error.
    I don’t see any other way to include
      all of the key aspects of fielding skill - positioning, quickness to the ball,
      and hands - in an analysis of fielding unless one starts with this
      premise.


  3. Fielders should not be penalized, or rewarded, for plays that are made by
      other fielders.
    I think most people would agree with this one.


  4. Fielders should be penalized when plays are not made that they could have
      made, given likely positioning of the defense.
    Note that I say “could have
      made,” not “should have made,” here, because in many cases we don’t know
      enough about the positioning of the fielders to make an accurate determination
      of responsibility for a ball in play. If all we know is that a team allowed a
      hit on a ground ball through the SS hole, we can’t assume that because the
      average 3B fields the ball 50%, the average SS fields it 40% of the time, and
      10% of the balls through the SS hole are not fieldable, that we should divvy
      up the responsibility that way. We don’t know enough to determine whether the
      3B or the SS, or both, or neither, had a chance to turn that specific ball
      into an out. We don’t know who was hitting, who was pitching, if the infield
      was up or back or at double-play depth, whether the SS had broken to cover
      second on a steal attempt, whether the 3B was up defending against a bunt
      attempt, or other variables that affect fielder positioning. All we know is
      that either the SS or the 3B could have been in position to make a play
      on the ball with the most likely alignments of the defense, and we also can be
      nearly certain that no other fielder could have been in position to make a
      play under likely alignments of the defense. We can reasonably conclude that
      both the SS and the 3B should be penalized because the play was not made, but
      we can’t determine which one was most responsible, or the extent to
      which either or both might have been responsible. The only sure way to ensure
      that both fielders receive their share of the responsibility is to treat both
      fielders as being fully responsible for missing the play.


  5. Fielders cannot be evaluated without accounting for the effects of team
      defensive context.
    Fielder positioning is driven by the distribution of
      balls in play against the team, and a fielder’s ability to cover a particular
      area of the field is, in turn, driven primarily by his positioning. Therefore,
      a SS who handles 45% of the balls in play in the SS hole behind the pitching
      staff of the Yankees can’t reasonably be assumed to handle 45% of the balls in
      play in the SS hole behind any other pitching staff, or (more importantly) 45%
      of the balls in play in the SS hole behind a league-aggregate pitching staff.
      One must evaluate a fielder in the context of his team’s distribution of balls
      in play, because it is that distribution - no other - which defines his
      opportunities to make plays.

 

Mike Emeigh Posted: November 03, 2002 at 06:00 AM | 25 comment(s) Login to Bookmark
  Related News:

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. Bernal Diaz has an angel on his shoulder Posted: November 03, 2002 at 02:00 AM (#607027)
I agree with your stance on positioning as a major factor but I have a bit of a problem with point #2. How can a line drive hit, or a hit down the line or off the wall be the fielder's fault? Maybe I am reading it wrong. Please explain how oviouls base hits can be the fielder's fault and not the pitcher's.
   2. Mike Emeigh Posted: November 03, 2002 at 02:00 AM (#607028)
How can a line drive hit, or a hit down the line or off the wall be the fielder's fault? Maybe I am reading it wrong. Please explain how obvious base hits can be the fielder's fault and not the pitcher's.

My study of play-by-play data suggests that, in general, the only thing that pitchers affect is whether the ball is hit on the ground or in the air. They have little impact on either the direction in which the ball is hit or how hard the ball is hit, and there is almost no evidence that pitchers *pitch to the defense* in key situations.

Furthermore, the extent to which a base hit is "obvious" is usually apparent only in retrospect. If the third baseman is guarding the line, and/or two or three steps deeper than normal, that shot down the line that in other circumstances would be an "obvious" double becomes an out. If the left fielder is shaded toward the line, that rope toward the corner that certainly looks like an "obvious" double is catchable. And balls that hit high enough off the walls to be truly uncatchable yet remain in the field of play are a relatively rare occurrence (except for places like Fenway with a tall wall or scoreboard somewhere, and even then the number of such balls aren't close to being a majority of the hits in play).

What I am suggesting is that we don't know a priori whether a particular ball that remains in the field of play could have been converted into an out, and furthermore that any assumption that we make about the nature those balls in play risks excluding important information about fielder positioning or quickness to the ball from our measurement approach. Thus, in order to make sure that we capture *everything* we need to know about fielding skill, we need to include *every* ball that remains in the field of play in our measurement system - even with the risk that we will unfairly penalize a fielder in that (relatively rare IMO) instance where a ball couldn't have been caught under *any* circumstance.

-- MWE
   3. Bangkok9 eschews 1 from Column A Posted: November 04, 2002 at 02:00 AM (#607030)
Mike, props to you for undertaking this study. I'm anxious to see how a few things are handled.

1) Any plans for special handling of fielding by pitchers? While I agree with what I take to be your premise that no part of the infield is indefensible, pitcher fielding may constitute a special case because of the short reaction time on hard-hit balls.

2) Are you planning on discounting as noise all of whatever small percent of balls to the outfield are indefensible? Or will there be a different treatment for balls off the wall at impractical heights vs. balls-not-off-walls hit to the various 'death valleys' [with right-center in Pac Bell Park as a leading example]?

I think the latter distinction might be important at some threshold. Consider that PBP's death valley is theoretically defensible but in a practical sense attempting to do opens up the gap in left-center with any reasonable configuration of outfielders. Only a generation of Shinjos would suffice...

Gratuitous Borges quotes aside, one could posit that any ball hit into a death valley for a hit is an X-error on the manager or general manager for improper construction of line-up or roster.

Looking forward to future installments!
   4. MattB Posted: November 04, 2002 at 02:00 AM (#607034)
Does your "Point 4" mean that a ball that is scored E-5 will be partially charged against the shortstop, just because it is hit between second and third?
   5. Mike Emeigh Posted: November 04, 2002 at 02:00 AM (#607037)
"We can reasonably conclude that both the SS and the 3B should be penalized because the play was not made, but we can't determine which one was most responsible, or the extent to which either or both might have been responsible. The only sure way to ensure that both fielders receive their share of the responsibility is to treat both fielders as being fully responsible for missing the play."

In doing this, don't you assure that they are both receiving more than their share of responsibility?


Yes, but because we don't have (and can't get) information on fielder positioning before the play, any assumption that we make about fielder responsibility is likely to be wrong in a way that excludes key information about positioning or ball distribution. I'll talk about this more when discussing zone-based rating systems.

BTW, insurance companies do assume that both drivers are responsible when specific fault cannot be affixed for an accident - each driver's insurance picks up the tab for that driver's car, and both driver's rates go up :)

By the way, I'm sure you realize that in choosing Derek Jeter as your primary example, you've ensured that very few people will be able to discuss your results objectively or without name-calling :) I wish you would have chosen a player about whom people have fewer preconceived notions, but I guess it wouldn't be as fun that way.

As the people who heard my SABR32 presentation know, I chose Derek Jeter precisely because his fielding numbers provide the best example of the problems with existing fielding analysis methods, because the underlying assumptions of those methods fail to confront the unique characteristics of the BIP distribution against the Yankees.

I subscribe to MLB.com's service that allows me to access "compressed games" - videos that clip out all of the footage except for those plays that result in an out, hit, error, runner advancement, etc. I When I watched clips of Yankee games this year, I was amazed at the number of balls that Ventura fielded at what was very nearly a "normal" shortstop position. I logged 5 such plays in the 20 or so Yankee games that I watched - I logged only one other such play by "any" other team in any of the other compressed game clips that I watched (and I watched a bunch of them - every one available from May through mid-July).

Any plans for special handling of fielding by pitchers? While I agree with what I take to be your premise that no part of the infield is indefensible, pitcher fielding may constitute a special case because of the short reaction time on hard-hit balls.

Few existing methods talk much about pitcher fielding; Clay Davenport, for one, deliberately chooses not to address it. Pitcher fielding skills do affect opportunities available to shortstops, obviously, but I'm not sure how significant the effect is. I will take another look at it in the PBP data, and when I discuss the PBP data in detail I'll address it there.

Are you planning on discounting as noise all of whatever small percent of balls to the outfield are indefensible? Or will there be a different treatment for balls off the wall at impractical heights vs. balls-not-off-walls hit to the various 'death valleys' [with right-center in Pac Bell Park as a leading example]?

In this series, I'm looking primarily at infield defense; the outfield BIP don't come into play. The PBP data from which I am working wouldn't allow me to make those distinctions anyway, and since my general rule is that it's better to make assumptions that are too broad rather than assumptions that are too narrow, I would treat all unfielded balls in play as plays that could have been made by fielders had they been in position to do so.

In case I haven't made it clear yet, I differ from most analysts in that I don't think it's possible, given our limited information on fielding, to make a supportable assertion that a fielder "should" have been able to make a play on a ball in play. The only assertion that we can support, based on the information available to us, is that a fielder "could" make a play on a ball in play in some situations, and unless we are absolutely certain that the BIP didn't occur in one of those situations, the fielder should be charged with the responsibility when the play is not made.

-- MWE
   6. Mike Emeigh Posted: November 04, 2002 at 02:00 AM (#607040)
As far as *1, that there are no routine outs or uncatchable balls, I don't see what the basis is for such an assumption.

The basis for such an assumption, as I've said before, is that we can't make that assumption a priori for a particular ball in play, absent information on the game situation and fielder positioning, and therefore we can't support an assertion that x% of BIP will *always* be routine outs or uncatchable balls. By excluding balls from the analysis beforehand, we run the (IMO unacceptable) risk of excluding key information about fielder positioning and reaction time to the ball which will help us differentiate between fielders.

I am not saying that the fielders "should" field 100% of the balls that remain in the field of play. I am saying that from an analytical standpoint, we have no supportable basis for excluding any balls in play from our analysis, and if we want to be sure that we've captured all of the nuances of fielding skill in our analysis, we have to treat every ball that remains in the field in play as a potential out for someone. I don't see any other reasonable option, given the information that we have.

Does your "Point 4" mean that a ball that is scored E-5 will be partially charged against the shortstop, just because it is hit between second and third?

No. In this case we have information available to us in the play record that indicates that no fielder other than the 3B could have reasonably made the play; fielders shouldn't be held accountable for either plays made by other fielders or mistakes made by other fielders when the play record is clear. A SS wouldn't get credit for a 5-3 groundout, either. The fourth point applies to situations (almost always hits in play) where it is uncertain from the record we have available to us who could reasonably have made the play.

-- MWE
   7. Mike Emeigh Posted: November 04, 2002 at 02:00 AM (#607041)
It looks like what you are saying is that the pitcher automatically gets credit for an "out" when he allows a ball in play, and then if it is actually a hit or an error, the fielder(s) are held fully responsible.

Not at all. What I am saying is that, when we evaluate fielding, the fielders get credit for all of the successes AND all of the failures, regardless of whether the failures are scored hits or errors. I thought that was clear - generally when people have traditionally evaluated fielding, the fielders get credit for the successes and the portion of failures that are called errors, and my point there was to expand the argument to include other plays (e.g. hits in play) as failures. If that wasn't clear, I hope it is now :)

But it sure looks to me so far that you might be overstating the practical impact of *positioning*. Just because a fielder could, in principle and if he were psychic, be in proper position to field every ball, doesn't mean that such is a realistic basis on which to evaluate fielders in the real world.

It is more realistic, in my opinion, than any method which proposes to assign the responsibility for fielding chances based on assertions about the extent to which a fielder "should" have been able to make a play - the latter methods understate the practical effect of positioning, as well as the effects of ball distribution in play. As I intend to demonstrate later in the series, ball distribution in play varies significantly from team to team, with the Yankees having an especially peculiar distribution. As Don Malcolm's FAD analysis in the 1999 BBBA demonstrated, infielders have very limited ability to make plays on balls for which they have to move more than about a step in either direction - so that if you aren't positioned correctly "off the bat", based on your ability to read the likely location of the ball when it's put into play, you have little chance of making the play. Thus, where the fielder *should* be positioned - and the areas of the field that he *should* be reasonably expected to cover - is team-specific and situation-specific, and no static method can hope to account for all of the team-to-team and situation-to-situation variations. A ball in play becomes a routine out primarily because the fielders anticipated where the ball would be hit and handled it cleanly; a ball in play becomes an uncatchable hit primarily because the fielders could not anticipate that the ball would be hit there. I don't deny that, in many if not most such cases, the fielders *shouldn't* expect the ball to be hit in that vicinity - but I can't make that judgment without specific knowledge of the pitcher, hitter, game situation, ball-strike situation, pitch thrown, etc. that I don't have.

If you can come up with a supportable alternate thesis, I'd be happy to hear it. But I have to warn you that I've heard most of them before, and all of the ones that I've heard (especially in support of ZR/UZR-type rating) fall because they make assumptions about fielder positioning and ball distribution that can't be supported on the available evidence.

-- MWE
   8. mike green Posted: November 04, 2002 at 02:00 AM (#607047)
Mike E,

Regarding your comment that there are no obvious outs. It seems to me that there is a difference between popups and ground balls. Popups are missed at about the same rate that balls out of the park are caught (save for Torii Hunter and Eric Davis). Catching a popup is, to my mind, no more a fielding achievement than failing to catch a home run is a fielding failure.

Distinguishing between popups and flares might be a challenge, but certainly outs after the IF fly rule is called, should be excluded from any calculation of both a fielder's opportunites and successes.

I agree with your other points.

Mike G
   9. Walt Davis Posted: November 04, 2002 at 02:00 AM (#607050)
Maybe after seeing play-by-play data I'd realize why this doesn't make sense or maybe I'm missing something else but...

As far as your problems with UZR, your points seem valid. If a certain ball is fielded 60% of the time by the SS, and the only reason Jeter only gets to that ball 45% of the time is that he properly positioned himself 2 steps in the opposite direction because of the Yankess' (or this particular pitcher's) ball distribution, then that is something which should be a part of the analysis.

Yes, no, and maybe. Seems to me that positioning just comes out in the wash. If Jeter gets to 15% fewer balls in the hole because Ventura (and Brosius before him) has great range, then Jeter should be positioned more towards 2B which means he should get to a higher percentage of balls up the middle. If Jeter is positioned closer to the hole than he needs to be, then this is his or the coaches' fault, but its impact is that he'd be a worse SS. In other words, if Jeter "gets to" 15% fewer balls in the hole and the average # of balls up the middle, then either (1) Jeter has lousy range or (2) Jeter is positioned in the wrong spot. If he "gets to" 15% fewer balls in the hole but 15% more up the middle, then he either has average range or he's well-positioned.

To put it another way, if your average SS can field balls 20 feet to his left and 20 feet to his right (i.e. range), then his positioning should just determine the placement of balls he can get to. If you've got a slow 3B and a pitching staff that gives up a lot of grounders to the left side, then you want to position him within 20 feet of the hole. If it's the other way around, you put him within 20 feet of second. The first SS gets to a higher %age of balls in the hole, the second one gets to a higher %age of balls up the middle.
   10. Walt Davis Posted: November 04, 2002 at 02:00 AM (#607053)
Sorry for being unclear. I wasn't necessarily speaking in regarding to ZR or similar such stats. Whether what I said applies precisely to those measures I can't say. I'm just talking in a theoretical fashion.

For yet another bad analogy, position is the mean of the distribution and range is the standard deviation. One wants to position one's fielders such that their distributions don't overlap with one another but do overlap substantially with balls in play. To a certain level of crudeness (i.e. zones defined as narrowly as possible) it seems one ought to be able to define a given player's distribution and from that estimate (average) position and range.

But like I said, I've never seen these data so for all I know I'm talking out my ass completely.
   11. Rob Wood Posted: November 04, 2002 at 02:01 AM (#607054)
Back to point #1 for a moment. I think that there are a lot of ground balls that are the equivalent of pop-ups. Every professional player from rookie ball to the major leagues would make the play 99.9% of the time. These are medium-to-slowly hit two or three hop ground balls that kinda hang in the air waiting for an infielder to easily make the play (hit not too hard and not too easy). Whatever positioning and reaction time of virtually any body would allow the play to be made easily (assuming that they can throw to first base semi-accurately).

Expanding on Walt's earlier point, it seems to me that if you have enough fielders and enough softly hit balls (suppose only pitchers batted, say), then the incorporation of the existence of routine outs into the analysis would be important.

Are you saying that the fielding data that we have cannot distinguish these "easy outs" from hard-hit ground balls that present a challenge no matter where the fielders are positioned? If not, then I understand where you are headed. If so, then I am puzzled.
   12. Mike Emeigh Posted: November 04, 2002 at 02:01 AM (#607055)
Maybe that is correct in the area of fielding, but the burden of proof is on you, I think, since the general presumption is just the opposite--that an estimate of something is better than not trying to estimate it at all.

We have two options. We can assert that all balls that remain in the field of play can be converted into outs. We know that is almost certainly untrue, but we at least know the nature of the error for all fielders (we are underrating everyone). If we're making an error in the same direction for everyone, the chances are that we'll have the right rank order.

Or we can assert that *x* percent of balls in play are unfieldable, and should be excluded from the analysis because no one could get to them and we unfairly penalize players. The logical question snow are how certain are we that *x* reduces the amount of error in our estimates, and how certain we are that we have the right rank order when we are done. I am suggesting that the error that we introduce when we try to make an estimate of *x* cannot be evaluated, and that furthermore when we try to make that estimate of *x* we can no longer be reasonably certain that the error we introduce will not change the rank order of the fielders - IOW, Fielder A may rank ahead of fielder B "because" of the way that we chose to estimate the percentage of unfieldable balls. If we can't validate our estimate of *x* with empirical evidence (which I believe we cannot with existing evidence), we increase the likelihood that we've got the order wrong.

can you tell us in advance how many compressed games you watched, and the distribution amongst teams

I watched every compressed game from May 1 through the All-Star break. The Yankees did make more of them available, to be sure, but there was a reasonable cross-section of all of the other major league teams, usually about 5-10 games per team (except for Florida and Montreal, who didn't have a lot of TV).

If you have figured out how to properly make such adjustments from the PBP data, then I look forward to seeing it.

It's my belief that there IS no such way to make "proper" adjustments to ZR/UZR data. Zones of coverage are not only dynamic, but they change from team to team, pitcher to pitcher, and game situation to game situation. I don't see any way to distill a common baseline from those varying contexts against which fielders can be evaluated without introducing simplifications that distort the ratings.

Are you saying that the fielding data that we have cannot distinguish these "easy outs" from hard-hit ground balls that present a challenge no matter where the fielders are positioned?

The extent to which a ball is an "easy out" depends on (a) the type of ball that is put into play, (b) the hardness with which it is hit, and (c) the relative position of the fielder to the ball. There is quite a bit of evidence that suggests that when major-league pitchers face major-league hitters, the pitcher has very little control over the latter two factors. Don Malcolm's 1997 FAD (Force and Distance) study suggested that *both* of the latter two factors were critically important - if the ball is hit more than one step away from the fielder it becomes significantly less likely to be an out no matter how hard it is hit - and that evidence in turn suggests that if the fielder moves even one step in the wrong direction the play is no longer likely to be what we classify as "routine".

So yes, what I am suggesting is that we cannot make an a priori distinction between a routine out and a non-routine play, in the absence of specific information about fielder positioning. Fielder positioning is IMO an aspect of fielder skill (albeit one that, like hitting skill and pitching skill, benefits from solid coaching and scouting) that we want to measure, and we can best assure that we capture all the data that we need to measure it by treating every ball that remains in the field of play as a potential out, without trying to make a *degree of difficulty* distinction between them. Real difference between fielders will show up anyway, in my opinion, and we're not likely to create additional distortions by making assumptions that we can't validate in any reasonable way.

-- MWE
   13. Marc Posted: November 04, 2002 at 02:01 AM (#607057)
What about throwing?
   14. Doug Posted: November 05, 2002 at 02:01 AM (#607059)
I'm struggling with your principles and your measurement philosophy.

Regarding principles, you assert that every ball in play, if not turned into an out, is attributable to inadequate fielding, and that a specific fielder be fully accountable for any ball he could have fielded had he been positioned in a way which could be construed as "normal" positioning.

My problem with these principles is that the game just doesn't work that way. There are lots and lots of places to hit a ball that would never be covered by any fielder in any "normal" fielding position. For example, any soft (or not so soft) liner hit 8 or 10 feet over the infielders' heads and landing 30 to 50 feet (or so) behind them just aren't going to be caught by any fielder, ever - there are no "normal" fielding positions which cover that area of real estate. If a batter happens to hit one there, we say he was lucky or skilful (generally depending on how awkward or graceful his swing looked).

Your response to this is that since we can't know which BIPs are the unfieldable ones and which aren't, then we must count them all. Which just seems silly to me. Counting them all is just a specific example of an estimate of the fieldable BIPs, and almost certainly a not very good estimate. Why is this better than using a reasoned estimate based on whatever empirical data (however imperfect) are available?

Your response to estimation is that this will undoubtedly introduce error which, heaven forbid, might incorrectly rank specific fielders relative to each other. Two observations. One, the "count them all" approach is an estimate which will introduce its own error. Two, I would be amazed if you were to claim that your results show that DJ is unequivocally the nth best fielder in the AL, and that there is not the slightest possibility that he should be ranked n-1 or n+1. To me, that isn't the point. I just want to know whether the method suggests he is one of the best, one of the worst, or middle of the pack - I'm not going to place any stock in the precise postioning. So, to get back to the original point, I don't buy the argument against using well-reasoned estimates.

Of course, my objections may just melt away with your next instalment - but will be watching closely to see how you handle the data given the apparent implications of your principles.
   15. Mike Emeigh Posted: November 05, 2002 at 02:01 AM (#607060)
There are lots and lots of places to hit a ball that would never be covered by any fielder in any "normal" fielding position. For example, any soft (or not so soft) liner hit 8 or 10 feet over the infielders' heads and landing 30 to 50 feet (or so) behind them just aren't going to be caught by any fielder, ever - there are no "normal" fielding positions which cover that area of real estate.

OK, fair enough. Now let me ask some questions:

-- How many of these balls would you estimate there are per team/season?
-- Which teams give up more of them? Which teams give up fewer?
-- What kind of pitchers give up more of them? What kind of pitchers give up fewer of them?
-- What are the techniques that you would use to estimate the number of these balls that are put into play?
-- How would you verify that your estimate was of the right order of magnitude?

and finally:

-- What difference does it make if we consider such balls unfieldable? How much more accurate would it make the final ratings if we did so? Any idea?

The effect can't be very large - because if it were, teams would likely redeploy their fielders to cover those areas of the field. The fact that hitters are considered "lucky" when they happen to get one of these adds further credence to the idea that there aren't a whole lot of such plays. So what is the marginal gain from eliminating them from the analysis? Is there enough of a benefit to our analysis to warrant making the effort to count all of these, and eliminate them?

If you can make a case that:

(a) it is possible to develop a systematic approach to estimating the number of balls that cannot be caught under any circumstances, based on the available evidence;
(b) the estimate can be verified with a reasonable degree of accuracy; and
(c) the numbers are likely to big enough to have an impact on our fielding analysis

then you'll have a justification for doing it. I've looked at this from every angle I can think of, and I don't think it's possible (with the second item being the sticking point, in my opinion), but I'm willing to listen if someone wants to make the case.

-- MWE
   16. Bernal Diaz has an angel on his shoulder Posted: November 05, 2002 at 02:01 AM (#607064)
One other thing about point #2 that ties into positioning. How are shifts for players like Thome and Bonds going to be taken into effect? Suppose Bonds or Thome hits ball towards 3rd that if a shift had not been on would be an easy out? But with a shift goes through as a hit? Also how are bunts going to be taken into account? And has been mentioned before what about throwing? Chuck Knoblauch a few years ago wasn't a bad fielder, he just couldn't throw.
I am eagerly awaiting your work. I think that quite a few of the so called Web Gems are a reslult of poor positioning rather than superior fleilding skills. Omar Vizquel is the player he is partially because he positions himself so well according to the bater and the pitch called. He does showboat a bit however. IMO most of the "great" diving outfield plays are not as a result of great feilding but as a result of poor routes to the ball and poor positioning that make require a diving catch. Lance Berkman is a perfect example as is Bernie Williams. As for Derek Jeter, for what he lacks in range he needs to make up for with positioning. He needs to study the tendencies of the pitcher vs batter situation more carefully. The play he made in the 2001 playoffs at the plate was pure luck.
   17. TeddyA Posted: November 05, 2002 at 02:01 AM (#607066)
I look forward to the rest of the series. I've always been troubled by the analyses that show Jeter to be an extremely poor fielder. This is not because I'm a Jeter fan but because of Bayesian analysis -- which, given my background I think of in terms of diagnostic testing in medicine. If a long term cigarette smoker with a lung mass has a biopsy that doesn't show cancer, I say: "we need to do another biopsy." This is because the pretest likelihood of cancer was >90% and the negative biopsy only reduces that to 50%.

Our "pretest" position about the defensive skill of the shortstop for a historically successful team, like the 1996-2002 Yankees, is that he is very good defender. If our "diagnostic test" -- be it zone rating or Win Shares -- yields a result that is a very poor fielder, then we should wonder about the efficacy of our analysis. This is particularly true when our mode of analysis is not thoroughly tested -- as is the case in all fielding analyses.

Analyses, like diagnostic tests, may be generally accurate but breakdown and give inaccurate results in particular cases either from chance or systemic problems. I look forward to reading about why analyses that I've found quite convincing may have been wrong about Jeter.
   18. TeddyA Posted: November 05, 2002 at 02:01 AM (#607067)
I look forward to the rest of the series. I've always been troubled by the analyses that show Jeter to be an extremely poor fielder. This is not because I'm a Jeter fan but because of Bayesian analysis -- which, given my background I think of in terms of diagnostic testing in medicine. If a long term cigarette smoker with a lung mass has a biopsy that doesn't show cancer, I say: "we need to do another biopsy." This is because the pretest likelihood of cancer was >90% and the negative biopsy only reduces that to 50%.

Our "pretest" position about the defensive skill of the shortstop for a historically successful team, like the 1996-2002 Yankees, is that he is very good defender. If our "diagnostic test" -- be it zone rating or Win Shares -- yields a result that is a very poor fielder, then we should wonder about the efficacy of our analysis. This is particularly true when our mode of analysis is not thoroughly tested -- as is the case in all fielding analyses.

Analyses, like diagnostic tests, may be generally accurate but breakdown and give inaccurate results in particular cases either from chance or systemic problems. I look forward to reading about why analyses that I've found quite convincing may have been wrong about Jeter.
   19. tangotiger Posted: November 05, 2002 at 02:01 AM (#607068)
Great comments all-around.

but I can't make that judgment without specific knowledge of the pitcher, hitter, game situation, ball-strike situation, pitch thrown, etc. that I don't have.

This is perhaps the most important statement I think Mike is making. Let's suppose that the chances of converting a ball in play into an out is influenced by
- the fielder,
the pitcher,
the pitcher handedness,
the batter,
the batter handedness,
the GB/FB tendecies of pitcher/batter,
the count,
the score,
the inning,
the base-out,
the other fielders,
the turf,
the park,
day/night,
the speed of the ball hit,
the initial positioning of the fielders,
the change in positioning during a pitch, etc, etc
we would all be happy.

I think, if we knew all that, we would answer with a great deal of confidence that an average SS, given the exact same conditions (as noted above) that Jeter faced, would have made "x" amount of outs, and that Jeter made "y", and therefore, I know Jeter's value.

So, wouldn't the first step be to analyze each of these conditions to determine the extent to which they are an influence? And if you decide that you will live with only ball location (UZR) and batter handedness, then this would imply a range of +/- 10 balls / 162 games = 1 standard deviation (or some such number)? But that if you include the base-out, the SD goes down to 5 balls, etc, etc.

So, I applaud most of what Mike says. But the statement about "liability" I think is very much open to debate.

   20. Bernal Diaz has an angel on his shoulder Posted: November 05, 2002 at 02:01 AM (#607071)
It is't a question of predicting 100% of the time where a ball is going to be hit. A SS can look in and see what pitch is going to be called and the location of that pitch. He then can cheat either to the left or the right depending on the tendancies of the hitter. Suppose this gives a SS a 5% better chance at getting to a ball. Over the course of the season this would mean that the SS got to perhaps 30-40 more balls than he would have had he not cheated. Is this significant? Of course it is.
It is all about reading pitcher vs batter tendancies and reacting to them. While you may not be able to be correct 100% of the time every little bit helps.
Now on the flip side a fielder may cheat and the ball will be hit in a location that he could have gotten it had he not cheated.
I am glad I am not taking this project on.
   21. User unknown in local recipient table (Craig B) Posted: November 05, 2002 at 02:01 AM (#607072)
Our "pretest" position about the defensive skill of the shortstop for a historically successful team, like the 1996-2002 Yankees, is that he is very good defender. If our "diagnostic test" -- be it zone rating or Win Shares -- yields a result that is a very poor fielder, then we should wonder about the efficacy of our analysis.

Ted, that's a good point, but it needs to be pointed out that in historical terms, Jeter is nearly off the charts as a hitter at shortstop. Jeter is one of the five or six best-hitting shortstops in the history of the game. It's hardly inconceivable that Jeter might be defensively poor and still make a very large net contribution to the Yankees' winning efforts.

Furthermore, while it is generally true that successful teams have had shortstops with good defensive reputations, this is not the same as saying they had good defensive shortstops. And it is not universally true. Jeff Blauser was Atlanta's shortstop through much of their 90s run; Blauser is little more than mediocre as a shortstop (as was his running mate Rafael Belliard). There are many oth

In the end, anyone arguing that Jeter is anything but a poor defensive shortstop has a huge information defecit to make up. There is absolutely no evidence, outside of the subjective testimony of the preponderance of the NY media and Tim McCarver, that Jeter is a good shortstop, and a large body of objective evidence, several analyses made in differing ways, that he is not. Once you have done your tenth biopsy, you should begin to suspect that there's no cancer there after all.

I am, I suppose, willing to be an agnostic about whether he's *good* defensively or not (though anyone watching this postseason saw lots of evidence he is not, the sort that those of us who go to AL parks see every year); he certainly has not ever been valuable defensively. If Jeter were a valuable defensive shortstop he would be helping his team to record outs as compared to other shortstops. He doesn't.
   22. User unknown in local recipient table (Craig B) Posted: November 05, 2002 at 02:01 AM (#607073)
I think the whole idea of 'positioning' as a way to determine a player's defensive ability is often vastly overstated by lots of statheads.

We would have that in common with players and managers, who generally when asked consider positioning one of the (if not the) most important of defensive skills. Cal Ripken, Mark Belanger, and Ozzie Smith, and especially Willie Mays, all well-known as defensive players, have spoken at length in several publications about the primary importance of good positioning. Managers, GMs, and scouts swear by it... ask Doug Melvin, ask Tony LaRussa, or just go down to the local park and ask a scout.

The best succinct statements on the importance of defensive positioning are those of Mays in Chapter 3 of Leonard Koppett's _Thinking Fan's Guide To Baseball_. Interestingly, Koppett calls Jeter's defense "brilliant" in that chapter.
   23. Mike Emeigh Posted: November 05, 2002 at 02:01 AM (#607074)
By disregarding "unfieldable" balls, you're implicityly assuming that the number of such hits is the same for all pitchers and all teams.

No. I'm suggesting that the difference between teams at a position in the number of balls that are "uncatchable" are likely to be smaller than the error we will introduce by trying to estimate how many of them there are.

How big do you think the range between the best team and the worst team is in this category? It clearly cannot be more than 200 or so balls (around 5% of the average number of BIP per team), because that is (roughly) the difference between the total number of H-HR allowed by the best team and the number of H-HR allowed by the worst team. When you figure in that (a) not all of those balls can be considered "unfieldable", and (b) a fair number of those "unfieldable" balls would not be considered opportunities for the SS even if you counted them - they'd go to other fielders - how big *can* the difference be at a particular position? I don't see how the range of performance from team to team could be more than 20-30 balls at *any* position, and that's top-to-bottom, best-to-worst, not the average difference team to team. If you think that's likely to have a big impact on your methods when looking at performance over 500 or so opportunities at a position per season, well...

-- MWE
   24. Doug Posted: November 05, 2002 at 02:01 AM (#607075)
-- How many of these balls would you estimate there are per team/season? -- Which teams give up more of them? Which teams give up fewer? -- What kind of pitchers give up more of them? What kind of pitchers give up fewer of them? -- What are the techniques that you would use to estimate the number of these balls that are put into play? -- How would you verify that your estimate was of the right order of magnitude?

OK Mike, good questions, but I think there should be ways to answer them in a reasonable way. And, gut feel, I just can't imagine that the unfieldable BIPs are negligible.

You alluded in your piece to the fact that your study makes use of Play-by-Play data. I haven't ever seen these data but, from what I've read in other pieces, included are, among other things, a location code to which every BIP is hit - and it's pretty precise, breaking the field down into several dozen distinct zones. So, if you've got that data, you can tell, pretty much straight-away, in which zones BIPs are seldom turned into outs, and in which zones outs come much more readily. Seems like a reasonable basis for coming up with estimates of how many BIPs really should be considered fieldable.

To take it a step further, if the play-by-play data tells you what zone a BIP was hit to, it probably also tells you who fielded it. So that should also give you a start on coming up with estimates on expectations for which fielder is more likely to field a ball in a certain location (or, to put it another way, to come up with a fairer way to establish expectations on which fielder should handle which BIPs).

If I'm wrong about the play-by-play data, and none of the stuff I've talked about exists, then please diregard my points. But I do think it exists somewhere, based on other pieces I've read on this site. And, obviously, I'm only talking conceptually here and it would undoubtedly be a mountain of work to account for all the nuances, but my point is I think it's quite possible to come up with reasonable answers to your questions.

I guess you can tell I've got no problem with doing estimates. My view is I think it's utterly impractical to assume that methods used to try to answer meaningful questions about the game will ever be 100% accurate 100% of the time. Just isn't going to happen. So, instead, why not try to make the best insights you can based on whatever data you've got? Having said that, I completely respect that you may have a different philosophical bent on this.
   25. Stephen Posted: February 22, 2004 at 04:09 AM (#614710)
I say who cares its baseball, to tell how good a player is by messing with numbers is crap. By watching players and how they preform their tasks you can tell how good one is...If someone is good their obviously good if their not their not, to go into percentages and numbers like that is just flat out crap and should not be done...its baseball...

You must be Registered and Logged In to post comments.

 

 

<< Back to main

BBTF Partner

Support BBTF

donate

Thanks to
Don Malcolm
for his generous support.

Bookmarks

You must be logged in to view your Bookmarks.

Syndicate

Page rendered in 0.6483 seconds
66 querie(s) executed