Is Colin’s method better after two seasons? Three seasons? Five seasons?

I'll repeat my concern from the thread on Wyers' previous column: After three seasons, won't a player's natural defensive ability change significantly? Doesn't that, then, make defensive metrics non-descriptive said player's natural ability?

Sounds like quantum mechanics.

TDF is right. If one really had to wait five seasons before saying with confidence that one fielder was better than another, it's not going to make any difference except in retrospect.

Or in other words, any scout with a clipboard could tell you a lot more than this metric about what shortstop to sign for next season.

After three seasons, won't a player's natural defensive ability change significantly?

That's true for batting and pitching and baserunning, too, but we still find a lot of meaning in measuring players' abilities in those parts of the spectrum over periods ranging from three years to a 20-year career.

This is not a solution (yet, by any means) for accurately quantifying a player's true fielding ability at any given moment, but it's a necessary and very important step on that road. Before, we had established that there was bias, but the only tool we had to try to measure the bias was a blunt one in WOWY. This is a sharper tool in the tool chest for measuring bias. Once we can measure the bias, we can start to eliminate it or adjust for it.

The new metric itself doesn't provide much predictive value. It seems to be good for historical analysis, but not for predicting upcoming seasons, especially for players who haven't played much in the majors yet. Maybe this metric could be adapted to the minors.

It seems to be good for historical analysis

And I should have added, not that there's anything wrong with that. Especially if a player's reputation doesn't seem to match what he accomplished over a given 5-year span. Do some organizations/managers judge fielders better than others? is the kind of question this could help explore.

Or in other words, any scout with a clipboard could tell you a lot more than this metric about what shortstop to sign for next season.

What about the scout in KC who thought Betancourt was good?

I don't think he had a clipboard. Etch'n'Sketch maybe.

After three seasons, won't a player's natural defensive ability change significantly?

That's true for batting and pitching and baserunning, too, but we still find a lot of meaning in measuring players' abilities in those parts of the spectrum over periods ranging from three years to a 20-year career.

But we're lead to believe that a season's worth of data for a pitcher or hitter can be somewhat predictive. Every defensive metric I've seen discussed talks about the need for 2-3 years of data to tell us anything. If it takes that much time to accumulate enough data on a fielder for it to tell us anything, how can any of the metrics tell us anything about a player's defensive ability today?

I haven't read the article yet but ...

neither bias nor variance is the concern in this situation but rather the mean squared error (MSE) or similar measure of total error. You will generally be happy to trade a little bias for a big gain in variance reduction. The point estimates aren't that big of a deal and it's more important how confident you are in saying "this guy is close enough to average to not worry about."

Now, not having read the article, I don't know how you judge bias in this situation anyway. We have no true value at the individual level to compare our estimate to (except perhaps plays made). We have some team-level measures (runs allowed) but that needs adjustment for pitching.

My guess is you guys want to look into techniques like "generalized regression" which is a rather simple but odd technique used for "benchmarking" or "calibrating" survey weights (so you need to look into sample survey statistics to find it) so they add up to population totals using various optimization criteria. I'm still not sure that will work.

If it takes that much time to accumulate enough data on a fielder for it to tell us anything, how can any of the metrics tell us anything about a player's defensive ability today?

It's not a question of how long it takes to tell us anything. Even one play tells us something, just not very much. It's a question of how much error you're willing to tolerate in your measurement.

My statement (really a WAG at this point) was that 2-4 seasons of Colin's system would have equivalent error to UZR or Plus/Minus or (insert your favorite advanced fielding metric here), and that for larger samples, Colin's system would probably be better, and for smaller samples, UZR or whatever might be better. I was not saying anything with respect to how many seasons of fielding data you would need to get a good read on a fielder's ability.

Fielding measurements are hard to make, and Colin's system is an important step on the road in trying to improve them. I did not make any claim, nor did Colin, as far as I know, that his system is better or worse than a scout's measurements. On that subject, though, a few thoughts:

1. What's the average error in runs on a scout's estimate of fielding ability?
2. Which scouts are better than others at estimating fielding ability?
3. How many scouts' opinions do you have access to for any given player? I have access to none, outside of something like the amateur scouting from Tango's Fans Scouting Report, which may be quite valuable. I'd love to have more of that information, but I think it's a fair assumption that we as a public community are never going to have access to that information. We would still like to estimate fielding ability, though, wouldn't we?

You will generally be happy to trade a little bias for a big gain in variance reduction.

That's the crux of the ongoing discussion on this topic. The working assumption had been that UZR and similar systems introduced little or no bias in exchange for a big gain in variance reduction.

Colin's work here is very helpful in quantifying the size of the bias. I estimated, for instance, in the case of Ozzie Smith, that the range bias in systems like TotalZone was on the order of 5 runs per season. That's where I came up with my guess that the tradeoff in variance reduction was a losing cause at somewhere around and after the 3-season mark. That's a very rough number, not to be taken as some sort of gospel, but that's the kind of thing that Colin's work is moving us toward.

What about the scout in KC who thought Betancourt was good?

Michael Kay was AMAZED by Betancourt's play a few days ago. Ken Singleton wagged his head in agreement.

I still don't understand why nobody uses a stopwatch to measure the time of flight of a ball in the air, and then note that on the grid depicting where the ball landed (be it in a fielder's glove or on the ground). For infielders you could time a grounder to the point it crosses the baselines, tho slow rollers still need to be quantified somehow (not clever enough to figure out how).

I still don't understand why nobody uses a stopwatch

I'd use a video frame-counter off a real-time replay. But yes, it could maybe work.

My understanding is that BIS is tracking time of flight for air balls.

Im surprised BP published anything by him figured the Weiters PECOTA card thing killed any chance of that

No article available only behind a pay wall can be "one of the most important pieces written about fielding measurement."

Again, I do not say they aren't allowed to do as they choose; of course they are. But scholarship can't be taken seriously unless it's universally available for review, critique, and response.

scholarship can't be taken seriously unless it's universally available for review, critique, and response.

So I guess nothing ever published only in a book or a journal can be taken seriously as scholarship, since by definition none of it is universally available.

So I guess nothing ever published only in a book or a journal can be taken seriously as scholarship, since by definition none of it is universally available.

You can always access these via a library somewhere. Maybe not your local one, but somewhere.