Melky +15 and Damon +9? Maybe Zito won't be so bad, just keep Matsui the hell off the field.

we ran into a huge problem with all batted balls last year for pop-up hogs (poster boy: Orlando Hudson), and it looks like this year our outlier is Albert Pujols.

I assume the Hudson effect is what causes Conor Jackson to look so bad according to PMR (-19).

Yeah, I hadn't really thought about Conor Jackson, but that has to be it. Zone rating had him as average, and -19 runs is, well, a lot.

Anyway, there are some stark differences on some players (especially Fenway outfielders) between PMR and ZR, so if anyone has any ideas on what might cause those differences, well, that won’t hurt the discussion.

Does BIS treat balls off the wall the same way Stats does? I would bet that PMR assumes balls off the wall are uncatchable, which should (and does) make Manny look a little bit better than ZR does.

I'm also curious if the strange dimensions in CF hurt Crisp more in ZR than it would in PMR or UZR.

Oh, and before I forget, thanks for doing the conversions.

Does BIS treat balls off the wall the same way Stats does?

I believe so.

-- MWE

I believe so.

***

Yeah, but Pinto doesn't, since the park is a separate parameter.

Theoretically, PMR should account for the Green Monster, where ZR does not.

Does anyone know if David's using the "smoothed visitor" model from last year or the original model?

Does anyone know if David's using the "smoothed visitor" model from last year or the original model?

According to some of the comments on the first set of corrected numbers for 2006, he's using the "smoothed visitor" model.

As I understand it, Pinto runs a regression for each type of BIP that yields a probability of it being converted to an out. But does anyone here (DSG, LAWBH?) have a sense of how narrowly defined each type of BIP is, i.e. how big is the sample for each type. Is the typical regression being run on 20 BIP, 50, 200? Since this is now using only 2006 visitor data -- essentially half of one season -- I could see the N getting pretty small on some of these, depending on how narrow the definition is. On the other hand, if a BIP bucket is defined too broadly, you would obviously lose a lot of information.

Also, is park one of the independent variables, or does he run separate regressions for each park? If it's the latter, then I'd think the sample size could really become problematic.

Yeah, I hadn't really thought about Conor Jackson, but that has to be it. Zone rating had him as average, and -19 runs is, well, a lot.

Conor is probably not nearly that bad, remember, he's playing next to last year's popup hog. I wouldn't be surprised though if his GB only rate is worse than his ZR - he didn't come to the majors with a rep for glovework.

Why are routine popups that turn into outs counting against (or for) anyone? Cant this be adjusted for?

APNY, this is guy's idea and illustration (that he posted over on Tango's site), which I think is the elegant one:

But let’s say the probabilities were:
1B out .60
2B out .25
RF out .10
Non-out .05

Pinto would give these net ratings if 2B makes play:
1B -.6
2B +.75
RF -.1

Whereas I’d give +.05 to whomever makes the play, and treat it as an opportunity only for the player who makes the play. But if no out is recorded, then punish them all proportionately:
1B -.6
2B -.25
RF -.1

So Coco Crisp is a +15 defender per PMR, while UZR and Dial have him in the -15 area. That's a hell of a swing...

But both systems agree that Ryan Howard is a terrible, terrible fielder. Makes me feel just a little worse for Pujols.

What's with the long-winded Phillies fan? Albert Pujols has never lacked range at any position and was not any more error-prone than most young 3Bs, after which he has settled down and become a very steady player. And then his second point:

it's true that howard's range is a bit less than average at first base, but so what?

Well, it would appear that Ryan Howard was roughly 60 outs worse than Albert Pujols defensively, so that's what.

How do these numbers compare to Dewan's, which also use BIS data? Just from looking at the limited release in the Bill James Handbook there are some sizeable differences.

Also, is park one of the independent variables, or does he run separate regressions for each park? If it's the latter, then I'd think the sample size could really become problematic.

***

It's not really an independent variable or a regression. It's a parameter. In a regression, we would say, p(out) = a*par1 + b*par2 + c*par3..., whereas in Pinto's model, we say, given par1, par2, par3...what is the probability of an out? But yes, the sample size is going to be pretty small for some zones. This is why MGL has talked about building a regression model for estimating probabilities in UZR for years. It just isn't very easy, if at all possible.

Crap, I was hoping this was about Pabst Mauve Ribbon - the Pabst for the ladies!

As I understand it, Pinto runs a regression for each type of BIP that yields a probability of it being converted to an out. But does anyone here (DSG, LAWBH?) have a sense of how narrowly defined each type of BIP is, i.e. how big is the sample for each type. Is the typical regression being run on 20 BIP, 50, 200? Since this is now using only 2006 visitor data -- essentially half of one season -- I could see the N getting pretty small on some of these

I do not think Pinto uses a regression. I believe that he calculates the expected outcome for a given set of parameters - e.g for a medium hit popup to vector K by left handed batter off right handed pitcher in fenway park - if out of 5 balls matching those conditions there were 2 outs by SS, 1 by LF, 1 by CF, 1 falls in, then per ball the expectation is 0.4 outs by SS, .2 each for LF and CF. The smoothed visiting model of PMR doesn't exclude the home team's performance but does severely discount it, so that visitors are more heavily weighted. Sample size issues are severe with one year of data. There are 60 possible parameter combinations for each vector. Consider that (just using the 22 fair STATS vectors, not the finer distinctions of BIS) there are an average of only 200 balls in play along a particular vector in one season in one park, which have to be further subdivided among the 60 combinations of batter handedness/pitcher handedness/hit type/whether the ball was hit soft, medium, hard. The smoothed visiting team model exacerbates the sample size problem.

Joe and DSG: thanks for the clarification.

It sounds like he'd be much better off combining the data for all 30 parks, increasing sample size be 30x, and then making park adjustments where necessary. For starters, you should probably treat GBs the same in all parks. It may not be quite true, but any difference you see in one year of data could as easily reflect the home hitters as a true park effect. Same for popups, at least those in fair territory. Dealing with LDs and FBs is harder, but maybe a park adjustment made at the vector level -- rather than each unique set of parameters -- could do the job.

I like PMR because it says Coco is good and I like Coco. However, if it says Jeter is good too, I won't like it so much.

But both systems agree that Ryan Howard is a terrible, terrible fielder. Makes me feel just a little worse for Pujols

No, one system shows them to be almost identical (Chris Dial's) and the other shows Pujols to be 50 runs better. I find it hard to believe any two ML 1B can be 50 runs apart in defense.

Stuff like this makes defensive statistics hard to accept.

Stuff like this makes defensive statistics hard to accept.

Biggest problem with defensive stats for me. These wild variations always throw methodology into question. Since this area is not my forte', has there been any of the studies that validate the results of another? If I'm seeing Dial's numbers and PMR numbers that far apart, I'm hard pressed to buy into one system or the other till the gap is breached.

The smoothed visiting model of PMR doesn't exclude the home team's performance but does severely discount it, so that visitors are more heavily weighted.

This, to me, is a handicap, as it runs the risk of overvaluing defenders who play positions that are difficult in their home parks. For instance, last season, each of the primary Minnesota outfielders did better under the smoothed visitor model than the original model. The ceiling of the Metrodome can be somewhat hazardous to fielders who aren't used to it, I would imagine, which matches (but certainly doesn't by itself prove) my hypothesis.

This, to me, is a handicap, as it runs the risk of overvaluing defenders who play positions that are difficult in their home parks. For instance, last season, each of the primary Minnesota outfielders did better under the smoothed visitor model than the original model.

This is a risk. But on the other hand, ALL above-avg fielders should do better under the visitor model (since they are being compared to themselves much more in the original model), so maybe the MN OF's are just above-average. I think it's more important to avoid the compared-to-themselves problem.

Intererstingly, this probably means that Manny is even WORSE than his PMR rating!

However, JoeArthur makes the point that the smooth visitor model makes all players look better than they are, since it predicts fewer outs than are actually made. I do think Pinto should adjust the expected outs to equal actual outs for each position and/or each vector.

Biggest problem with defensive stats for me. These wild variations always throw methodology into question. Since this area is not my forte', has there been any of the studies that validate the results of another? If I'm seeing Dial's numbers and PMR numbers that far apart, I'm hard pressed to buy into one system or the other till the gap is breached.

***

Maury, you might be interested in this: http://stats.mostvaluablenetwork.com/general/a-detailed-comparison-of-defensive-metrics/

The issue with this version of PMR is that it uses pop-ups (and liners) for infielders, which zone rating does not include, and no defensive metric should include (at least not the way PMR does it). So it's not the same statistic. It would be like saying, "I don't trust these batting stats when some players have a very different number for batting average and on-base percentage." The two statistics both purport to measure offense, but they actually are measuring different things. PMR and ZR both purport to measure defense, but they too are using different statistics.

this probably means that Manny is even WORSE than his PMR rating!

With regard to the smoothed visitor model specifically, this is probably true. Of course to reach this conclusion one msut assume that Manny is indeed below average! And/or that the average visitor is average or below average, rather than above average. Playing in the AL East, the average visiting left fielder at Fenway in 2006 included a healthy dose of Carl Crawford, Reed Johnson and Nick Markakis, who seem to be pretty good.

However, apart from sample size issues, there are a couple of peculiarities about Manny and PMR.
1) The conventional wisdom around Boston is that Manny should do better at Fenway with its smaller effective area to defend, and his presumably limited range would be more exposed in road games. But in 2005 David Pinto published PMR home/road splits for Manny which were hard to understand. [I'm going by memory since his site is down at the moment, but I think he had Manny +3 on the road and -15 at home in 2005.] Manny is very deferential when another fielder can make a play, so he is eligible for a severe ball hog penalty under PMR [I think this explains a lot of Coco Crisp's plus rating in center field as well]. With essentially no outfield foul territory to protect, Fenway encourages the left fielder to shift toward left center a little, increasing opportunities for overlap with the centerfielder. My guess is that this, rather than luck alone, explains much of the wide gap between his home and road performance under PMR.
2) To avoid have separate models for infield and outfield in PMR, Pinto does not use batted ball distance as a parameter. Without having seen the BIS data directly to know how they actually assign the "soft" and "hard" hit attributes, I imagine that this can mean that a medium fly ball to vector "I" could travel 260 or 310 or 360 feet, and PMR would treat all the balls as having the same chance of being caught. So because of this there's a great possibility of noise in 1-year PMR outfield ratings. And in fact data I have assembled for 2006 suggests that much of the difference between Red Sox LF and their opponents occurred along vector "I", but that the average distance of balls hit to the outfield in that direction by the red sox was about 302 feet, and only 285 feet by opponents. Other things being equal (which I don't know yet), the visiting team's opportunities along that vector should have been easier to catch, and I don't think PMR would recognize that.

Intererstingly, this probably means that Manny is even WORSE than his PMR rating!

Actually, last year he did better in the original model than the smoothed visitor model. I think Fenway LF is unique, but not necessarily difficult in the sense positions in other parks might be.

The shortstop figures are up, and so are my conversions.

If he puts up any over the weekend, I won't be doing any conversions, as I'll be away from my spreadsheets.

as I'll be away from my spreadsheets.

The saddest day for any saberist.

:)

I may run into Pinto today if he shows up at the SABR regional. Did John Dewan come out with #s for 2006?

Did John Dewan come out with #s for 2006?

I think he had top 10s in the Bill James Handbook, but I haven't heard of any full list like in the Fielding Bible.

Right fielders here. Also, the first attempt to use distance rather than velocity to rate outfielders.

I think he had top 10s in the Bill James Handbook, but I haven't heard of any full list like in the Fielding Bible.

Thanks, Dan. ZR, UZR, PMR, Fielding Bible, Davenport, Range, Win Shares. Are there any systems that I'm missing?

Here are the RF conversions. The figures using distance agree much more with zone rating than the original PMR formulation. There is in both methods a huge disagreement on Brian Giles. Petco's rightfield is what I hypothesize to be the culprit.

BTW, he's got the 2B stuff up, but I don't know if I'll get to it today. Maybe this afternoon.

The 2B conversions.

It doesn't look like the Orlando Hudson popup stuff has as big an effect this year, though he did not rank well by ZR, so I wonder if the difference between his ratings by those two methods may be attributable to balls in the air.

LAHBW,
those are your runs, right? Not mine? I thought I have Griffey as worse than -11.

The issue with this version of PMR is that it uses pop-ups (and liners) for infielders, which zone rating does not include, and no defensive metric should include (at least not the way PMR does it).

I'd rather include information than exclude it. Liners should be included, because line drives caught goes to positioning, which is an important aspect of defense. As for popups, I think that the number of truly discretionary popups is overestimated and that the primary fielder can be identified for most of them. on a popup, the fielder with the best angle will typically be the one who makes the play. On most short infield popups near the pitchers' mound, the 3B makes most of them on the left side, the 1B makes most of them on the right side. On most infield popups near 2B, the fielder who has to backpedal the least typically makes the play - if the 2B is coming at an angle while the SS is mostly going back, the 2B will normally catch it.

-- MWE

Hmmm, does Pinto just do starters? I was looking for Kevin Youkilis's numbers in LF and couldn't see them.

Hmmm, does Pinto just do starters? I was looking for Kevin Youkilis's numbers in LF and couldn't see them.

He uses a cut-off of 1000 BIP.

The players who don't reach that cutoff appear to be collectively well below average - I think LAWoBH had said the shortstops are apparently 97 runs below average as a group.

He uses a cut-off of 1000 BIP.

The players who don't reach that cutoff appear to be collectively well below average - I think LAWoBH had said the shortstops are apparently 97 runs below average as a group.

OK.

LAHBW,
those are your runs, right? Not mine? I thought I have Griffey as worse than -11.

You did have him at -11 for actual runs, it's -18 over 150 games. I'm comparing the actual numbers as the per-150 I do for the PMR conversions is an estimate.

Pinto got tothird base last week.

"I like PMR because it says Coco is good and I like Coco. However, if it says Jeter is good too, I won't like it so much."

Well, per the link in post 30, it doesn't like Jeter much. Clearly there's a Red Sox bias in these numbers.

Though it does put Alex Gonzalez lower than many Sox fans would have had him for last season. (My own sense is that his defense suffered due to injury in the second half.)

DSG did some comparison last year (I think) of PMR, Dial's system, and other defensive systems. IIRC, Dial and PMR had similar correlations with mgl's UZR. How consistent are Dial and PMR year to year? I imagine that the better, more useful, metric would have more internal consistency.

I'd rather include information than exclude it. Liners should be included, because line drives caught goes to positioning, which is an important aspect of defense.

I don't think so at all. Mostly because all you track are ones that are caught, and the distribution is completely random. Including them is wrong - it completely ignores chances.

As for popups, I think that the number of truly discretionary popups is overestimated and that the primary fielder can be identified for most of them.

Not really, because the *pitcher* is the primary fielder. There are ALWAYS two fielders who can make the play on IF popups.

on a popup, the fielder with the best angle will typically be the one who makes the play. On most short infield popups near the pitchers' mound, the 3B makes most of them on the left side, the 1B makes most of them on the right side. On most infield popups near 2B, the fielder who has to backpedal the least typically makes the play - if the 2B is coming at an angle while the SS is mostly going back, the 2B will normally catch it.

While it is true that is who *usually* makes the play, the problem is you cannot distinguish that very well in the data. Popups can almost always be caught by 2 or more fielders, and thus should be excluded.

I got the 3B up,y'all.

Dale Arnold mentioned PMR on his radio show with Michael Holley on WEEI. I did a doubletake.

Pitchers, Catchers, and Firstbasemen are up.

Now if only Pinto would do the GB only results for the infield.

I'm busy with work and trying to marry Erin Andrews, but I'll try to get the 1B conversions this afternoon. I don't do conversions for P and C.

Found a partial answer to my question upthread:

DSG did some comparison last year (I think) of PMR, Dial's system, and other defensive systems. IIRC, Dial and PMR had similar correlations with mgl's UZR. How consistent are Dial and PMR year to year? I imagine that the better, more useful, metric would have more internal consistency.

3. LA Waterloo of BH, Registered Hollywood Contrarian Posted: January 26, 2006 at 07:45 PM (#1839788)

There were 27 CF’s that appeared on last year’s and this year’s PMR list. Using the Original PMR Method (as opposed to the Smoothed Visitor Method Pinto presents as an alternative), and using the conversion to Runs Per 4000 BIP, I get Excel gets a correlation of .42509 for the group between the years ... I don’t really know how that compares to anything else in the world.

Were there any changes to firstbasemen, BHW?

Link to next year's thread.