Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Dialed In > Discussion
Dialed In
— 

Monday, December 31, 2007

Defensive Analysis - Continuous Improvement

Retrosheet has become even more valuable?  Who would think that was possible?  They are also becoming faster, and Baseball Reference is also reaching new heights in capabilities and faster updates.  Coupled with Chone Smith’s skill-sets, I have managed to look at last year’s defensive ratings.

At the beginning of this past baseball season, Chone took a look at OCab’s defensive plays, and pondered the results.  Why didn’t OCab’s chances match up with what his Zone Rating, my personal favorite defensive system, claimed he made?  I have studied ZR for a long time.  I am certain that it had been defined as “ground balls into a player’s zone converted into outs, as a fraction”.  Chone’s analysis demonstrated this to not be true.  Line Drives (LD) caught appeared to be included.  It didn’t make sense to me – I had asked this question specifically of STATS before and was told that LDs were not included.  Really, though, the math was not making sense either way.

So I asked the inventor, John Dewan.  John truthfully answered, “I’m not sure.”  Fair enough – the designer doesn’t always have all the controls over the actual inputs.  John now does (and designed) the work at BIS, and brilliant work it is.  His new system uses slightly different zones than ZR, and properly separates “Balls In Zone” (BIZ) from “Out Of Zone” plays made (OOZ).  This has always been recognized as a flaw in the ZR system, causing players who live at the fringes of their assigned zones to be over/underrated.

At any rate, a long discussion ensued regarding the number of plays an infielder made, and whether or not LDs were included.  Steve Treder interjected, “Including line drives won’t likely have a big effect, but it would seem to have a net useful effect. Why not include them, even if it only helps a little?”  Which was largely ignored, as we continued to argue semantics/minor points.  This discussion was the first and only known occurrence here at BTF.

Long story short (I know, too late), Retrosheet has the 2007 data, and I kindly asked Chone (and Mike Emeigh) to pull that data (because I suck at that).  Then I plugged the results in my standard DPI spreadsheets to calculate Defensive Runs Saved (DRS) by ZR, and then by ZR with LDs removed.

Chone is working on additional defensive analyses that are going to be awesome.  What I am going to work on is making sure there is a good and practical way for you, the Consumer, to accurately calculate defensive value for yourself.  I think that’s important.  Chone is going to do much fancier work, and he’s doing work to simplify the explanations, and I think it is going to shine the light on how defense can work.  But enough about how awesome he is.

So back to the discoveries from this pbp data.  I calculated DRS+ (adjusted to a standard baseline) for each player as I always have, and in the same manner anyone with access to ZR can, whether from ESPN or MLB or whathaveyou.  Then I took every player’s individual plays on the infield, separating the ground balls (GB), the line drives and the pop ups (PU).  It was pretty easy to then use the existing ZR to back-calculate how many ZR chances each player had excluding line drives, and calculate a LD-less ZR.  Then using this new ZR, calculate the DRS+.  It really isn’t too fancy.  After doing that, I could simply subtract my LD-adjusted ZR DRS+ from STATS ZR DRS+, and generate the difference including line drives can make.

The effect of including LDs is minimal, and negligible, in my opinion, effectively validating the original work above.  It’s essentially like using Marcel versus running lots of projection data work to improve the rating by a single run – or less.  The maximum DRS+ change listed below is the absolute value.  A player could have had his DRS+ affected negatively or positively by his LDs caught as compared to average.  The average DRS+ change listed below is for the starters – those playing 700 innings or so.  If I include all defenders at a position, the average drops to nearly nothing.

Lge  Pos  Max    Avg
AL   1B    1.2    0.3
AL   2B    1.5    0.4
AL   3B    2.1    0.7
AL   SS    1.1    0.5

NL   1B    1.3    0.6
NL   2B    1.9    0.9
NL   3B    2.1    0.7
NL   SS    1.5    0.8

In general, players with poor DRS+ have their numbers improved by removing the LDs, and the better fielders have their DRS+ decreased by removing the LDs.  That makes sense – good defenders are slightly better at snaring LDs, and the factor that, as the average decreases, the worse players are closer to average.  They “regress to the mean”, as it were.

The conclusion is that STATS ZR, while including LDs, does not unfairly malign players, or wrongly represent their defensive contributions.  As Treder said well: why not include them?  They are plays made, and the tightness around the run value indicates they are very close in opportunity and conversion.  Yes, I personally think LDs shouldn’t be included, but let’s not throw the baby out with the bath water.  You have a good working method to provide yourself with a good estimate of a player’s defensive value you can calculate all season, at any time, by yourself.  Hopefully this extra information allows you to be more confident in that data.

Chris Dial Posted: December 31, 2007 at 04:11 AM | 25 comment(s) Login to Bookmark
  Related News:

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. Chris Dial Posted: December 31, 2007 at 06:24 AM (#2657179)
Gold Glove Awards to follow.
   2. GuyM Posted: December 31, 2007 at 04:01 PM (#2657264)
In general, players with poor DRS+ have their numbers improved by removing the LDs, and the better fielders have their DRS+ decreased by removing the LDs. That makes sense – good defenders are slightly better at snaring LDs, and the factor that, as the average decreases, the worse players are closer to average.

I'm not sure this explanation is right. We'd expect removing the LDs to narrow the variance because a good LD rating raises the overall DRS+, and vice-versa. That will be true even if the LD outs are totally uncorrelated with fielding ability on GB. It seems to me it's that correlation -- GB-only rating and LD outs -- you want to look at. If there is a correlation -- positive or negative -- then you probably want to include LDs. If there isn't, then snagging LDs is probably mostly random and including them just reduces the accuracy of the metric (though apparently not enough that it matters a whole lot).
   3. GGC don't think it can get longer than a novella Posted: December 31, 2007 at 04:23 PM (#2657281)
I'm not the sharpest tool in the shed, but it seems to me that the more info you have thhe better. I am not running a team nor do I play rotisserie nor am I in a contemporary DMB league. I'm more interested in what a guy did than what he probably will do.
   4. Mike Emeigh Posted: December 31, 2007 at 04:26 PM (#2657284)
It seems to me it's that correlation -- GB-only rating and LD outs -- you want to look at.


The problem with doing this is that LD outs are such a small percentage of total outs that the correlation (or lack thereof) probably isn't meaningful amidst the noise. Only about 20% of BIP are line drives, and only about 20% of line drives are caught - you're talking about around 1 LD out per team/game.

-- MWE
   5. GuyM Posted: December 31, 2007 at 04:41 PM (#2657296)
The problem with doing this is that LD outs are such a small percentage of total outs that the correlation (or lack thereof) probably isn't meaningful amidst the noise.

You can fix that by looking at multiple-season and/or team data. Do infields that handle GBs well also record more LD outs, in general?

it seems to me that the more info you have thhe better.... I'm more interested in what a guy did than what he probably will do.

This gets said a lot, but the question is whether it's really "info." In this context, it's only info if it tells us whether the fielder is better or worse than the average player at his position. And without knowing the opportunities, just knowing the outs recorded may not tell us that. If the variation in LD opportunities each season is greater than the variation in the skill at catching them -- which seems pretty likely -- then adding this "information," paradoxically, reduces your actual knowledge.
   6. Mike Emeigh Posted: December 31, 2007 at 04:57 PM (#2657322)
Do infields that handle GBs well also record more LD outs, in general?


Infields that handle GBs well tend to be infields that face a lot of them - and infields that tend to face a lot of them tend to see fewer LDs, and the ones they do see tend to be harder to catch (because they're more likely to be off mistakes by the GB pitchers).

I'm still of the opinion that much of what we credit to fielders is actually pitching staff effect.

-- MWE
   7. GuyM Posted: December 31, 2007 at 05:15 PM (#2657339)
Infields that handle GBs well tend to be infields that face a lot of them - and infields that tend to face a lot of them tend to see fewer LDs, and the ones they do see tend to be harder to catch (because they're more likely to be off mistakes by the GB pitchers).

All LDs are "mistakes" :>). Is there actually evidence that the LD out% is lower behind GB pitchers? That's interesting. But I have to think the effect is small, if it exists.

I'm still of the opinion that much of what we credit to fielders is actually pitching staff effect.

Do you know of systemic impacts that zone/PBP data does not pick up? For example, do GB pitchers give up easier-to-field GBs even after controlling for location and batter handedness? (Of course, it could also be true that teams with GB staffs actually hire better infielders on average, so even this is hard to disentangle.)
   8. GGC don't think it can get longer than a novella Posted: December 31, 2007 at 05:34 PM (#2657361)
In this context, it's only info if it tells us whether the fielder is better or worse than the average player at his position.
Wouldn't this be akin to throwing out balls in play when evaluating pitchers? Sure, that might be helpful for projecting pitchers going forward, but it doesn't necessarily tell us who was a better pitcher that year.
   9. GuyM Posted: December 31, 2007 at 05:51 PM (#2657377)
With fielding, there isn't a bright line between assessing actual current value and projecting future performance (as there pretty much is with hitting), because the data doesn't tell us definitely who it was that performed the accomplishment. Was the LD caught because the pitcher allowed a LD hit right at the fielder, or did the fielder catch a ball that most other fielders would have missed?

For pitchers, we include BIP because we believe (and have evidence) that a good or bad result usually has more to do with the distribution of BIP allowed than the quality of the fielders. Even though that distribution itself reflects a lot of luck (see DIPS), we feel the pitcher should get credit/blame for that outcome because he threw the ball and it's not clear who else we could implicate. Also, to the extent we can adjust for the quality of his fielders, most analysts would agree this would give us a more accurate read on a pitcher's real performance. Certainly Jim Palmer, to take an extreme case, should be evaluated in the context of having an extraordinary defense behind him most of the time.
   10. Mike Emeigh Posted: December 31, 2007 at 10:18 PM (#2657645)
For example, do GB pitchers give up easier-to-field GBs even after controlling for location and batter handedness?


GB pitchers have higher out rates on GBs than do FB pitchers in general, and a lower rate of outs on FBs. I take that as strong evidence that they are in fact giving up easier-to-field GBs, but proving that isn't especially easy.

Of course, it could also be true that teams with GB staffs actually hire better infielders on average, so even this is hard to disentangle.


Or it could be true that we think they are good infielders BECAUSE they play behind GB staffs :)

-- MWE
   11. GuyM Posted: December 31, 2007 at 10:22 PM (#2657650)
GB pitchers have higher out rates on GBs than do FB pitchers in general, and a lower rate of outs on FBs. I take that as strong evidence that they are in fact giving up easier-to-field GBs, but proving that isn't especially easy.

Yes, that's true. But the issue here is whether the zone data captures that. If so, it isn't a problem. But if systems like Dial's, UZR and Plus/minus show infielders doing better than average when a GB pitcher is on the mound, then there's a problem.
   12. Mike Emeigh Posted: December 31, 2007 at 10:29 PM (#2657658)
But if systems like Dial's, UZR and Plus/minus show infielders doing better than average when a GB pitcher is on the mound, then there's a problem.


ZR (which is the basis for Dial's system) did when I last looked at it. UZR in its original conception did, although I don't know if MGL's changes have addressed that. PMR did as well. I haven't had time to pull the data to look at plus/minus.

-- MWE
   13. Chris Dial Posted: January 01, 2008 at 09:58 PM (#2658010)
But if systems like Dial's, UZR and Plus/minus show infielders doing better than average when a GB pitcher is on the mound, then there's a problem.

The significance of the problem also matters. Yes, it's a flaw with ZR that LDs are included. But as we can see, the flaw is very small.

I suspect we'll see the same.

What happens with GB pitchers and fielders is that most BIP in zones are "easily turned into outs". That's *why* the zones are what they are. GB pitchers throw more GBs, thus more GBs in zones. Of course it will result in more "easy" pays. I don't see where this is a significant problem.
   14. Mike Emeigh Posted: January 01, 2008 at 10:16 PM (#2658017)
GB pitchers throw more GBs, thus more GBs in zones. Of course it will result in more "easy" pays. I don't see where this is a significant problem.


The implication of defensive rating systems is that "higher percentage of plays made" is supposed to mean "better defensive quality". If a fielder plays behind a GB staff that gives him a higher percentage of "easy plays" than a fielder on another team, does the fact that he converts a higher percentage of them mean he's a better defender? I'd certainly wonder whether the first defender would continue to make a higher percentage of plays than the second defender if they swapped teams - and should there be a situation where they DID swap teams, and the first defender fell behind the second defender, I'd certainly wonder whether we were measuring pitching staff effects instead of defensive skill.

-- MWE
   15. Chris Dial Posted: January 01, 2008 at 10:39 PM (#2658022)
The implication of defensive rating systems is that "higher percentage of plays made" is supposed to mean "better defensive quality". If a fielder plays behind a GB staff that gives him a higher percentage of "easy plays" than a fielder on another team, does the fact that he converts a higher percentage of them mean he's a better defender? I'd certainly wonder whether the first defender would continue to make a higher percentage of plays than the second defender if they swapped teams - and should there be a situation where they DID swap teams, and the first defender fell behind the second defender, I'd certainly wonder whether we were measuring pitching staff effects instead of defensive skill.

As I said, you're going to have to find teams with a regular large disparity. I don't doubt there is some effect - is it significant?
   16. AROM Posted: January 02, 2008 at 03:58 AM (#2658096)
That makes sense – good defenders are slightly better at snaring LDs, and the factor that, as the average decreases, the worse players are closer to average.


From the data we have, we can't tell how good any infielder is at catching line drives, just how many linedrives he caught. The data I sent Chris was just the number of linedrive outs each infielder caught by year. We have no idea how many opportunities they had.

If an infielder starts with 320 plays made in 400 chances (.800), and 20 of those are line drives, his zone rating without linedrives is going to be 300/380 (.789) and with linedrives backed out of the league, he'll be compared to a slightly lower league average. So its not surprising to see it makes very little difference in his rating.

I'm assuming that the only line drives zone rating counts are the ones caught - if linedrives missed are counted in his opportunities then we just don't have the data to tell how much impact line drives are making on zone rating.

ZR (which is the basis for Dial's system) did when I last looked at it. UZR in its original conception did, although I don't know if MGL's changes have addressed that. PMR did as well. I haven't had time to pull the data to look at plus/minus.


MGL does have an adjustment for a pitcher's GB ratio.
   17. Slinger Francisco Barrios (Dr. Memory) Posted: January 02, 2008 at 04:13 PM (#2658228)
I'm more interested in what a guy did than what he probably will do.

That sentence separates the sheep from the goats quite nicely. I, too, am less interested in prediction than in who did what.
   18. Mike Emeigh Posted: January 02, 2008 at 04:38 PM (#2658243)
I'm assuming that the only line drives zone rating counts are the ones caught - if linedrives missed are counted in his opportunities then we just don't have the data to tell how much impact line drives are making on zone rating.


BIS requires its scorers to record the location of line drives based on where the ball lands. I assume that STATS does it the same way, and I suspect that line drives that land in-zone are counted as opportunities, but I have no way of knowing for sure.

-- MWE
   19. Chris Dial Posted: January 02, 2008 at 04:55 PM (#2658267)
When I was a scorer, and the instruction booklet for scoring, that a ball that lands in the zone is effectively a GB. I know that sounds odd, but the zones cover the dirt section, and while there may be some scored differently, I doubt it is significant.

I know we don't have a "chances" for LDs (nor do I think one would be accurate), but counting LD/inning gives us some idea of how well a player caught LDs. I think this is something ARod excels at, or did in 2007. A few more seasons (and AROM is obviously helping with that) and we'll generate some trend data for a given fielder, as well as any age or size relationships.
   20. JPWF13 Posted: January 02, 2008 at 05:01 PM (#2658274)
That sentence separates the sheep from the goats quite nicely. I, too, am less interested in prediction than in who did what.


Who are the sheep and who are the goats?

I am far more interested in prediction personally.
   21. Slinger Francisco Barrios (Dr. Memory) Posted: January 02, 2008 at 06:54 PM (#2658380)
Who are the sheep and who are the goats?

No pejorative was meant to be implied. Just there's two kinds of folks: those who care about predictions, and those who don't.
   22. GuyM Posted: January 02, 2008 at 08:29 PM (#2658484)
That sentence separates the sheep from the goats quite nicely. I, too, am less interested in prediction than in who did what.


This really isn't a very helpful distinction when evaluating fielding. By the time you've figured out who really did what, you've basically made a talent evaluation.
   23. John M. Perkins Posted: January 04, 2008 at 04:55 PM (#2660118)
Let's throw out cans of corn too.
One the scouting keys of 3B is can they handle the hot corner, not the lukewarm corner.
Cherry picking data to through out cheapens the analysis.
   24. Exploring Leftist Conservatism since 2008 (ark..) Posted: January 05, 2008 at 08:29 AM (#2660844)
Anyone know if anyone regularly analyses fielding by splicing together on video every play a given fielder is involved in?

Some MLB clubs must do this, no?
   25. Chris Dial Posted: January 09, 2008 at 02:43 PM (#2664203)
Arkitekton,
based on the GG analysis I went through, something is going on in Toronto. They've had lots of guys rate highly in converting balls to outs, so they, while not completely doing what you've said, done something to improve their fielders.

You must be Registered and Logged In to post comments.

 

 

<< Back to main

BBTF Partner

Support BBTF

donate

Thanks to
Francis
for his generous support.

Bookmarks

You must be logged in to view your Bookmarks.

Syndicate

Page rendered in 0.2586 seconds
68 querie(s) executed