Good stuff. Very readable, Justin makes understanding defence a lot easier (read his whole series on offensive evaluation as well).

no Dial?

I think you want RMSE here instead of correlation. UZR and ZR correlate quite highly but that doesn't necessarily mean that they are giving you similar +/- run estimates. This may be why he finds the two "careful" measures (UZR and PMR) don't correlate as well as UZR does with ZR.

For those not clear what I'm saying, the following two variables would be perfectly correlated (i.e. there is an exact linear relationship between them ... in this case Y=2X) despite not agreeing in magnitude:

X Y
1 2
2 4
3 6
4 8

But their RMSE (root mean squared error -- take the diff, square it, sum across all cases, take the mean) is 7.5.

The following two are not perfectly correlated but might be considered in more agreement:

A B
1 4
2 2
3 3
4 4

and they have an RMSE of 2.25. (I suppose technically that should be an RMS "difference" rather than "error" but whatever.)

It would also be interesting to look at the Spearman rank correlation among these series (i.e. the extent to which players are ranked in the same order on the measures) which, at some level, is what I think most folks would want.

There are a few outliers where UZR is far greater than ZR in magnitude, but most of the time its not that big a deal.

@Quinn--Thanks!

@Walt,

Thanks for the comments--it's a good point and I may give that a look as well. As AROM said, the differences aren't huge in terms of relative magnitudes of the different statistics, but it still might be worth a look.

Also, I did use Dial's method to convert ZR into a +/- runs statistic. So when you see ZR in that article, that's Dial.
-Justin

I would reinforce, however, that the finding of relatively low agreement between BIS-based data and STATS-based data is not new. Michael Humphrey's study from last August at THT showed surprisingly large apparent disagreements between data from the two providers, even after methodologies were standardized as much as possible. So I don't think that my finding of low agreement between PMR and UZR is completely due to how I chose to compare those numbers. And given that there's not really a great way to independently audit BIS vs. STATS, we probably need to be considering what both companies are telling us when making fielding evaluations.
-j

Well-organized piece, and important for the Reds, who, uhh, have had a few issues with their defense.