Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Baseball Newsstand > Baseball Primer Newsblog > Discussion
Baseball Primer Newsblog
— The Best News Links from the Baseball Newsstand

Wednesday, July 28, 2010

THT: Fast: A very important article on fielding

Baseball Prospectus has published an article by Colin Wyers today that may be one of the most important pieces written about fielding measurement in the last decade. The full piece is available only to BP subscribers, but let me briefly recap some of the topics Colin covers.

Colin reiterates the point that uncertainty in fielding measurements is something that can be tackled with bigger sample sizes, i.e., more season of data. Bias, on the other hand, is persistent. It does not decrease with larger sample sizes of fielding data. He mentions two types of bias: that related to park/scorer and that related to the fielder’s range.

He then outlines a clever method for using data like putouts and assists in order to develop a fielding metric for infielders that should be much less subject to those two sources of bias than our current advanced metrics like Ultimate Zone Rating (UZR), Plus/Minus, and TotalZone. His metric very likely has greater uncertainty than the advanced fielding metrics that use ball-in-play data to determine which fielder had the best chance to field a batted ball. However, at some point, larger sample sizes should decrease the effect of the uncertainty, such that the reduction in bias using Colin’s method will actually produce more accurate measures of fielding. Is Colin’s method better after two seasons? Three seasons? Five seasons? Because we don’t yet know the size of the park-scorer bias or range bias, we don’t know exactly at what point that occurs.

Thanks to Seth.

Repoz Posted: July 28, 2010 at 12:46 PM | 20 comment(s)
  Related News: GeneralSabermetrics

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

Page 1 of 1 pages
   1. TDF, situational idiot  Posted: July 28, 2010 at 02:10 PM (#3601707)
Is Colin’s method better after two seasons? Three seasons? Five seasons?
I'll repeat my concern from the thread on Wyers' previous column: After three seasons, won't a player's natural defensive ability change significantly? Doesn't that, then, make defensive metrics non-descriptive said player's natural ability?
   2. Al Kaline Trio  Posted: July 28, 2010 at 03:18 PM (#3601809)
Sounds like quantum mechanics.
   3. Bob Dernier Cri  Posted: July 28, 2010 at 03:25 PM (#3601814)
TDF is right. If one really had to wait five seasons before saying with confidence that one fielder was better than another, it's not going to make any difference except in retrospect.

Or in other words, any scout with a clipboard could tell you a lot more than this metric about what shortstop to sign for next season.
   4. Mike Fast  Posted: July 28, 2010 at 03:28 PM (#3601822)
After three seasons, won't a player's natural defensive ability change significantly?


That's true for batting and pitching and baserunning, too, but we still find a lot of meaning in measuring players' abilities in those parts of the spectrum over periods ranging from three years to a 20-year career.

This is not a solution (yet, by any means) for accurately quantifying a player's true fielding ability at any given moment, but it's a necessary and very important step on that road. Before, we had established that there was bias, but the only tool we had to try to measure the bias was a blunt one in WOWY. This is a sharper tool in the tool chest for measuring bias. Once we can measure the bias, we can start to eliminate it or adjust for it.
   5. Primakov throws at people in a professional manner  Posted: July 28, 2010 at 04:49 PM (#3601912)
The new metric itself doesn't provide much predictive value. It seems to be good for historical analysis, but not for predicting upcoming seasons, especially for players who haven't played much in the majors yet. Maybe this metric could be adapted to the minors.
   6. Bob Dernier Cri  Posted: July 28, 2010 at 04:57 PM (#3601924)
It seems to be good for historical analysis

And I should have added, not that there's anything wrong with that. Especially if a player's reputation doesn't seem to match what he accomplished over a given 5-year span. Do some organizations/managers judge fielders better than others? is the kind of question this could help explore.
   7. Johnny Sycophant-Laden Fora  Posted: July 28, 2010 at 05:10 PM (#3601938)
Or in other words, any scout with a clipboard could tell you a lot more than this metric about what shortstop to sign for next season.


What about the scout in KC who thought Betancourt was good?
   8. Fred Ludacris (Crispix Attacks)  Posted: July 28, 2010 at 05:35 PM (#3601965)
I don't think he had a clipboard. Etch'n'Sketch maybe.
   9. TDF, situational idiot  Posted: July 28, 2010 at 05:55 PM (#3601985)
After three seasons, won't a player's natural defensive ability change significantly?

That's true for batting and pitching and baserunning, too, but we still find a lot of meaning in measuring players' abilities in those parts of the spectrum over periods ranging from three years to a 20-year career.

But we're lead to believe that a season's worth of data for a pitcher or hitter can be somewhat predictive. Every defensive metric I've seen discussed talks about the need for 2-3 years of data to tell us anything. If it takes that much time to accumulate enough data on a fielder for it to tell us anything, how can any of the metrics tell us anything about a player's defensive ability today?
   10. Walt Davis  Posted: July 28, 2010 at 06:07 PM (#3601991)
I haven't read the article yet but ...

neither bias nor variance is the concern in this situation but rather the mean squared error (MSE) or similar measure of total error. You will generally be happy to trade a little bias for a big gain in variance reduction. The point estimates aren't that big of a deal and it's more important how confident you are in saying "this guy is close enough to average to not worry about."

Now, not having read the article, I don't know how you judge bias in this situation anyway. We have no true value at the individual level to compare our estimate to (except perhaps plays made). We have some team-level measures (runs allowed) but that needs adjustment for pitching.

My guess is you guys want to look into techniques like "generalized regression" which is a rather simple but odd technique used for "benchmarking" or "calibrating" survey weights (so you need to look into sample survey statistics to find it) so they add up to population totals using various optimization criteria. I'm still not sure that will work.
   11. Mike Fast  Posted: July 28, 2010 at 06:08 PM (#3601992)
If it takes that much time to accumulate enough data on a fielder for it to tell us anything, how can any of the metrics tell us anything about a player's defensive ability today?


It's not a question of how long it takes to tell us anything. Even one play tells us something, just not very much. It's a question of how much error you're willing to tolerate in your measurement.

My statement (really a WAG at this point) was that 2-4 seasons of Colin's system would have equivalent error to UZR or Plus/Minus or (insert your favorite advanced fielding metric here), and that for larger samples, Colin's system would probably be better, and for smaller samples, UZR or whatever might be better. I was not saying anything with respect to how many seasons of fielding data you would need to get a good read on a fielder's ability.

Fielding measurements are hard to make, and Colin's system is an important step on the road in trying to improve them. I did not make any claim, nor did Colin, as far as I know, that his system is better or worse than a scout's measurements. On that subject, though, a few thoughts:

1. What's the average error in runs on a scout's estimate of fielding ability?
2. Which scouts are better than others at estimating fielding ability?
3. How many scouts' opinions do you have access to for any given player? I have access to none, outside of something like the amateur scouting from Tango's Fans Scouting Report, which may be quite valuable. I'd love to have more of that information, but I think it's a fair assumption that we as a public community are never going to have access to that information. We would still like to estimate fielding ability, though, wouldn't we?
   12. Mike Fast  Posted: July 28, 2010 at 06:16 PM (#3602002)
You will generally be happy to trade a little bias for a big gain in variance reduction.


That's the crux of the ongoing discussion on this topic. The working assumption had been that UZR and similar systems introduced little or no bias in exchange for a big gain in variance reduction.

Colin's work here is very helpful in quantifying the size of the bias. I estimated, for instance, in the case of Ozzie Smith, that the range bias in systems like TotalZone was on the order of 5 runs per season. That's where I came up with my guess that the tradeoff in variance reduction was a losing cause at somewhere around and after the 3-season mark. That's a very rough number, not to be taken as some sort of gospel, but that's the kind of thing that Colin's work is moving us toward.
   13. Best Dressed Chicken in Town  Posted: July 28, 2010 at 07:04 PM (#3602038)
What about the scout in KC who thought Betancourt was good?

Michael Kay was AMAZED by Betancourt's play a few days ago. Ken Singleton wagged his head in agreement.
   14. John DiFool2  Posted: July 28, 2010 at 07:49 PM (#3602083)
I still don't understand why nobody uses a stopwatch to measure the time of flight of a ball in the air, and then note that on the grid depicting where the ball landed (be it in a fielder's glove or on the ground). For infielders you could time a grounder to the point it crosses the baselines, tho slow rollers still need to be quantified somehow (not clever enough to figure out how).
   15. tfbg9  Posted: July 28, 2010 at 08:10 PM (#3602095)
I still don't understand why nobody uses a stopwatch


I'd use a video frame-counter off a real-time replay. But yes, it could maybe work.
   16. Mike Fast  Posted: July 28, 2010 at 08:21 PM (#3602100)
My understanding is that BIS is tracking time of flight for air balls.
   17. Imeat  Posted: July 29, 2010 at 02:16 AM (#3602364)
Im surprised BP published anything by him figured the Weiters PECOTA card thing killed any chance of that
   18. Vaux, A.B.D.  Posted: July 29, 2010 at 04:16 AM (#3602379)
No article available only behind a pay wall can be "one of the most important pieces written about fielding measurement."

Again, I do not say they aren't allowed to do as they choose; of course they are. But scholarship can't be taken seriously unless it's universally available for review, critique, and response.
   19. rdfc  Posted: July 29, 2010 at 06:49 PM (#3603137)
scholarship can't be taken seriously unless it's universally available for review, critique, and response.


So I guess nothing ever published only in a book or a journal can be taken seriously as scholarship, since by definition none of it is universally available.
   20. fra paolo  Posted: August 01, 2010 at 09:24 AM (#3605408)
So I guess nothing ever published only in a book or a journal can be taken seriously as scholarship, since by definition none of it is universally available.

You can always access these via a library somewhere. Maybe not your local one, but somewhere.
Page 1 of 1 pages

You must be Registered and Logged In to post comments.

 

<< Back to main

Support BBTF

donate

Thanks to
SugarBear Blanks
for his generous support.

My Bookmarks

You must be logged in to view your Bookmarks.

Vivid Seats is a sports ticket broker, concert ticket broker and theater ticket broker offering the best baseball tickets like Yankees tickets, Cubs tickets, and Red Sox tickets, as well as Police reunion tour tickets and Jersey Boys tickets.

We have baseball tickets, the NFL schedule, college football tickets and Cowboys tickets. We have NBA tickets like Celtics tickets and Lakers tickets. Plus, buy concert tickets, Patriots tickets and Colts tickets. Also check out our MLB baseball schedule

Baseball Bats

 

 

 

Find Yankee collectibles and more

Major League Baseball: All Star Game, New York Yankees, Boston Red Sox, LA Angels, Washington Nationals, Chicago White Sox, and the Chicago Cubs.

Find amazing, cheap New York Yankees tickets for the 2010 season at Yankees Stadium, terrific Chicago Cubs tickets for Wrigley Field bleacher seats, Boston Red Sox tickets for Fenway and the Green Monster, and cheap tickets to any MLB baseball game.

Alliance Tickets has cheap tickets available to all MLB games. We also have tickets to major concerts and theater events. Get tickets to the Colorado Rockies, the Seattle Mariners and all your favorite baseball teams. We also carry tickets to all the major Sporting Events.

Page rendered in 1.2878 seconds
69 querie(s) executed