Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Baseball Newsstand > Baseball Primer Newsblog > Discussion
Baseball Primer Newsblog
— The Best News Links from the Baseball Newsstand

Wednesday, March 20, 2019

Reframing WAR at Fangraphs

Update: An earlier bug that impacted updated pitcher WAR has now been resolved. The pitcher tables below have been updated to reflect that. Thanks to everyone who pointed out the issue!

I’m very pleased to announce that FanGraphs has finally added catcher framing data to the site, with full thanks to Jared Cross, who you may know as the co-creator of the Steamer projections. We’ve also incorporated catcher framing into WAR.

Including catcher framing in WAR has been a topic of internal debate at FanGraphs for the past half-decade. The problem has never been with the inclusion of framing numbers on the catcher side of things. That’s a fairly simple addition. The problem has always been what to do with the pitchers. For instance, the 2011 Brewers were some 40 runs above average in catcher framing. When you add those 40 runs to catchers, do you subtract 40 runs from pitchers? As it turns out, you do, but those runs are not attributed equally to each pitcher

Bote Man Posted: March 20, 2019 at 03:17 PM | 184 comment(s) Login to Bookmark
  Tags: fangraphs, framing, pitch framing, war

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

Page 2 of 2 pages  < 1 2
   101. Kiko Sakata Posted: March 21, 2019 at 03:22 PM (#5824393)
KIKO: if you reading this: Can you explain how you calculate or where you find data on how many balls OF'ers can cut off and prevent an extra base hit?

I know you said you were measuring this on your statistical website, but i dont see that data anywhere and im not sure how you would estimate it.

thanks.


Okay, this is off-topic, but I am reading this. I don't exactly have that data; I infer it. My source is Retrosheet data. For balls-in-play, the pieces of information that I use are the end-result of the play (out, single, double, triple), the first fielder who fielded the ball (the guy who made the out for outs; the guy who fielded a hit for hits), and the hit-type (bunt, ground ball, fly ball, line drive). The last of these isn't consistently available before the 1980s; the middle one gets spotty, especially on hits, as you go back in history.

Retrosheet only have location data for about a dozen years (1988 - 1999, I think), so I can't use that directly. So, instead, I use it indirectly.

Anyway, for, say, line-drive doubles fielded by the left-fielder, I go back to the years for which I have location data, back out the implied distribution of locations that produced line-drive doubles fielded by the left fielder and impute the expected probability of such a play having been a double. If, say, all such plays had an a priori 40% of being doubles, then 60% of the double is debited to the defensive team; I split that between pitchers and fielders based on the extent to which player winning percentages persist in the particular component. (Also, a double that was fielded by the left fielder wasn't always the left fielder's fault; third basemen and center fielders also take some blame on these, I think.)

Sorry, I know that sounds overly complicated: mostly because it is.

So, basically, the underlying data is, how many doubles and triples and whatevers were fielded by a particular player. I have a bunch of spreadsheets - one per season - that contain this data buried within them. But I've never systematically pulled that particular information together, nor do I know of anybody / anyplace else who's done so. I'm not sure how complicated it would be for me to do so.

I do, however, have the ex ante probabilities by league-year. They're linked at the bottom of my "League" pages: here's 2018 - change the value after "y=" to get different years. There's a lot of numbers here, but, for example, to go back to a line-drive double fielded by the left fielder, such a play had an a priori probability of being a double of 34.4% with a 34.3% probability of it having been a single, a 0.8% chance of it having been a triple, and a 30.6% chance of it having been an out. If it's actually a double, the responsible fielders get "credit" for it not having been a triple (which you'd have expected on something like 1.2% of hits) and blame for it not having been a single.
   102. Rob_Wood Posted: March 21, 2019 at 03:34 PM (#5824395)
Can someone summarize/describe how the pitch-framing data is converted into runs/wins?

   103. Never Give an Inge (Dave) Posted: March 21, 2019 at 03:37 PM (#5824396)


Or general managers stopped hiring catchers who suck at framing and the standard deviation tightened.


Certainly possible. Would have to look at the data in aggregate and not just a few selected examples. If the change was more gradual, then you're probably right. If it was more abrupt, then I would say it's more likely to be due to a chance in officiating.
   104. DL from MN Posted: March 21, 2019 at 03:45 PM (#5824398)
I wish had bookmarked the article but I read something that said minor league catchers' impact on a called third strike was roughly 10%. Even if the catcher frames strike 3 perfectly he's only 10% of the reason it got called a strike. Anyone know what article I read?
   105. SoSH U at work Posted: March 21, 2019 at 03:46 PM (#5824399)
Given all the caveats, why does the umpire allow pitch framing to be a thing?


It's not consciously allowing anything. You have to make a call immediately on a ball that is moving, past a batter who may be swinging, with other things going on. If the pitcher hits the spot where the catcher is sitting, or if the catcher only moves a little to receive it, the pitch appears good. Many times, after you've called ball/strike, you think/know you might have missed it, but you can't just correct yourself. "Sorry, sorry, that was really a strike." That would be much worse.

You just go on.

I can say that one thing that doesn't affect me is the catcher pulling the glove back over the plate.

I don't know if this experience is similar with more experienced umps, such as Misirlou, but that's my take.
   106. DL from MN Posted: March 21, 2019 at 03:50 PM (#5824401)
If it was more abrupt, then I would say it's more likely to be due to a chance in officiating.


Or a change in measurement technology - Statcast in 2015.
   107. Misirlou doesn't live in the restaurant Posted: March 21, 2019 at 03:57 PM (#5824402)
I don't know if this experience is similar with more experienced umps, such as Misirlou, but that's my take.


Pretty much.
   108. Cleveland (need new name) fan Posted: March 21, 2019 at 04:00 PM (#5824403)
Allow?


What word is better? Umpires don't have to be influenced by the catcher, so its appears to me to be somewhat "voluntary" that pitch framing is an issue. I could be wrong, so that is why I asked SOSH for his insight.
   109. Never Give an Inge (Dave) Posted: March 21, 2019 at 04:08 PM (#5824404)

It would really be good to know how they apportion value for catcher framing. Are they looking at the Statcast data, evaluating the called balls and strikes for each catcher against what Statcast says, and then apportioning some value for each extra strike/ball?

Or are they doing something more akin to what is done with fielding stats -- i.e. for pitches that hit this part of the zone, X% are called strikes on average, this catcher had Y% called for strikes, so we credit him with Y-X additional strikes. And then apportioning some value based on that.

The second obviously would be the right way to do it.
   110. BillWallace Posted: March 21, 2019 at 04:08 PM (#5824405)
This makes me sad. I prefer Fangraphs' numbers in general but this seems like a huge mistake.

As with almost all of Sabrmetrics, the calculations involved make a number of underlying assumptions. Sometimes you may not even realize the assumptions that you're making. Often these assumptions are straightforward, and are likely to hold true in almost all cases. Sometimes they are not. If you get to the end of your calculations and the results you come up with don't make sense, strain credulity, don't pass the smell test... and in fact are directly contradicted by some back of the envelope math (i.e. the ERA comparisons done in this thread), then the chances are that you're relying on assumptions that are not actually true.

Here's an example of the type of assumption I'm talking about, and with the caveat that I don't actually know how they calculate these numbers. I think they calculate the value of a strike in the context of the count. So they're calculating the value of going from 0-0 to 0-1, but not confusing that with the value of going from 1-2 to K. HOWEVER even doing that, there is still an underlying assumption that going from 0-0 to 0-1 in all cases has the same value. It may very well be that going from 0-0 to 0-1 with a borderline strike that the ump quickly realizes was a missed call is substantially less valuable than going from 0-0 to 0-1 with a no-doubt strike.

Maybe this is a big effect, maybe it isn't, but the key is that they may not even realize they're making this assumption, and I doubt they're accounting for it. And if it turns out that it does matter, or other assumptions matter and are wrong, then it explains why their results are so non-credible.

I would hope that Fangraphs has further analysis to back up these results without just including them as is, because if they don't I think that's very very poor from them. I would have at least regressed them heavily, with a note about the uncertainty.

I don't believe these numbers at all.
   111. Cleveland (need new name) fan Posted: March 21, 2019 at 04:23 PM (#5824408)
I don't know if this experience is similar with more experienced umps, such as Misirlou, but that's my take.


Thanks for the insight. I am still a skeptic here though.
   112. BrianBrianson Posted: March 21, 2019 at 04:43 PM (#5824410)
What word is better? Umpires don't have to be influenced by the catcher, so its appears to me to be somewhat "voluntary" that pitch framing is an issue. I could be wrong, so that is why I asked SOSH for his insight.


The thinking underlying this is so bizarre I'm not sure I could even begin to guess at it. Like, have you ever met a human being before? Grocery stores play slow music so people will buy more, and they do, but I'd guess ~0 of them think it through as "Hmm, they're playing Kenny G & Celine Dionne, I better put a few more bags of flour in the cart".
   113. Never Give an Inge (Dave) Posted: March 21, 2019 at 04:50 PM (#5824412)

This, by the way, seems like the right way to compare two catchers' framing ability, and you could do something similar for each player relative to league average. Not sure if that's what FG does but it probably is, given that Johjima and Rob Johnson both show up as poor framers in the Fangraphs stats.

   114. . Posted: March 21, 2019 at 05:13 PM (#5824419)
Here's an example of the type of assumption I'm talking about, and with the caveat that I don't actually know how they calculate these numbers. I think they calculate the value of a strike in the context of the count. So they're calculating the value of going from 0-0 to 0-1, but not confusing that with the value of going from 1-2 to K. HOWEVER even doing that, there is still an underlying assumption that going from 0-0 to 0-1 in all cases has the same value. It may very well be that going from 0-0 to 0-1 with a borderline strike that the ump quickly realizes was a missed call is substantially less valuable than going from 0-0 to 0-1 with a no-doubt strike.


Exactly, and there's still the "sub-event" problem. Can a batter really "control" the caliber of pitches he sees? If not, why is he penalized if he sees better pitches? If a hitter sees pitches during the season that the league would hit .230/.310/.370 against and he hits .250/.330/.390, has he added value? If not, why not?

The answer with hitters is that we integrate all the sub-events into the final event, ultimately ignoring the sub-events. Is that the right approach? If we do that for hitters, why wouldn't we do it for catcher framing, too?

And then how does it make sense to add the catcher framing WAR, based on sub-events onto hitting WAR, based on final events? Answer: It really doesn't.
   115. JAHV Posted: March 21, 2019 at 05:13 PM (#5824420)
I agree with others that it doesn't pass the smell test, but even if it does, I love pitch framing. I love that it's a skill that differentiates catchers, and I love that catchers are getting credit for it, even if they're getting a little too much. This is coincidental, but a common complaint on catcher evaluation is that the counting stats, like WAR, don't value them properly because of their relative lack of playing time compared to other positions. Catch framing adjustments could add a bit of value to those guys and push them up near other players, career-wise.

Pitch framing skill is one of the reasons why I hope we never see an automatic strike zone. I think baseball is more entertaining with it being a skill.
   116. . Posted: March 21, 2019 at 05:15 PM (#5824421)
I would hope that Fangraphs has further analysis to back up these results without just including them as is, because if they don't I think that's very very poor from them. I would have at least regressed them heavily, with a note about the uncertainty.

I don't believe these numbers at all.


Fangraphs is in the business of convincing customers that it can continually come up with new baseball numbers to churn and analyze; thus, it very much serves its financial interests to act as if it can do that with catcher framing. In a very real sense, it's talking its book, making high skepticism very much in order.

If they're not even going to show their work, there's zero reason to pay attention.
   117. . Posted: March 21, 2019 at 05:23 PM (#5824424)
This, by the way, seems like the right way to compare two catchers' framing ability, and you could do something similar for each player relative to league average. Not sure if that's what FG does but it probably is, given that Johjima and Rob Johnson both show up as poor framers in the Fangraphs stats.


Agreed, and if you did it that way the differences would be marginal if not infintesimal -- as they were between Johjima, thought of as a brutal framer, and Rob Johnson.

That said, even this method has the "sub-event" issue. Why would we give a catcher credit for "gaining" a strike, but not a hitter?
   118. Kiko Sakata Posted: March 21, 2019 at 05:34 PM (#5824427)
Why would we give a catcher credit for "gaining" a strike, but not a hitter?


Because for hitters, it all comes out in the wash. Every sub-event is going to be allocated solely to the batter, because on the offensive side, who the hell else is going to get the credit, the third-base coach for flashing the take sign?

On the defensive side, the issue, presumably, is that the proper division of credit/blame between pitcher, catcher, and fielder(s) will vary across individual sub-event: called strike right down the middle - that's all to the pitcher; pitch that may or may not have just nicked the outside corner - this is where "pitch framing" wants to give some credit to catchers.

Line-drive double to the gap in left-center on a 2-2 count - for the batter, any change in value from having gotten the count from 0-0 to 2-2 is essentially wiped away by the final result - a double on a 2-2 count is the same as a double on an 0-1 count. But on defense, the idea is that maybe the catcher deserves some credit for getting the count to 2-2, which maybe reduces the probability of the batter hitting that line-drive double to the gap. So, on the defensive side, the pitcher/fielder(s) would presumably take a bigger debit than the batter gets a credit, but that's offset by the catcher getting a small credit (negative debit?) for working the count to 2-2, so that the sum of the offensive credits is equal to the sum of the defensive debits.

I'm not sure I buy this - i.e., I think your argument about diving down to the sub-event has some merit. And I definitely don't buy the actual magnitude of numbers shown by Fangraphs (and, to be fair to them, by others, too; everybody who's tried to quantify this has ended up in the same general ballpark). But that's why we would treat catchers and hitters differently at the sub-event level.
   119. . Posted: March 21, 2019 at 05:40 PM (#5824431)
Because for hitters, it all comes out in the wash.


It does for catchers, too. The "gained" strike is swallowed up by the final event with catchers just as with hitters. This reality is independent of the process of dividing up "credit."

Hitters: OK, great, you took a close pitch that 80% of the league would have swung at, giving you a more favorable count making the aggregates shift in your favor. But you ultimately struck out, so who cares?

Catchers: OK, great, you turned a close pitch into a strike that for 80% of the league's catchers would have been a ball, giving your pitcher a more favorable count and making the aggregates shift in his favor. But ultimately the hitter hit a home run, so who cares?

No reason to credit the catcher for anything -- whether shared or otherwise -- unless we're otherwise crediting hitters. Or so the argument would go.

Line-drive double to the gap in left-center on a 2-2 count - for the batter, any change in value from having gotten the count from 0-0 to 2-2 is essentially wiped away by the final result - a double on a 2-2 count is the same as a double on an 0-1 count.


Not sure about this one, comes down to philosophy. A double on an 0-2 count is certainly more impressive versus aggregate expectations than a double on a 3-1 count. Why no credit to the batter for that value add? Because it's his "fault" he got in the 0-2 count? Maybe. But what if the two pitches were both picture perfect? If he got into the 0-2 count on two 95 sliders called strikes on the black and then hit the double isn't that more impressive than hitting the double after taking two BP fastballs down the middle to go 0-2?

So when exactly are we supposed to just look at the final event and say "Forget all the rest" and when aren't we?
   120. Kiko Sakata Posted: March 21, 2019 at 05:48 PM (#5824434)
Why no credit to the batter for that value add? Because it's his "fault" he got in the 0-2 count?


Yes, precisely. Baseball statistics are mostly an accounting mechanism.

If he got into the 0-2 count on two 95 sliders on the black and then hit the double isn't that more impressive than hitting the double after taking two BP fastballs down the middle to go 0-2?


From my perspective, not really, no.
   121. . Posted: March 21, 2019 at 05:50 PM (#5824436)
Baseball statistics are mostly an accounting mechanism.


Sure, but some at the event level, some at the sub-event level.
   122. Karl from NY Posted: March 21, 2019 at 05:54 PM (#5824437)
There's no such thing as a sub-event level. Anything one would try to define as such could itself be defined as an event.
   123. Kiko Sakata Posted: March 21, 2019 at 05:54 PM (#5824438)
Sure, but some at the event level, some at the sub-event level.


I actually agree with you. If you're going to give catchers credit for "pitching" value, it should be done at the event level, not the sub-event level, and I suspect that's causing a lot of the problems with the implausible magnitudes here. I'm just saying: if you suddenly decided that you wanted to start doling out batting credit by pitch instead of by plate appearance, you'd just end up in the same place as before, because there's no one else to give the pitch-by-pitch credit to except the batter, and, as an accounting mechanism, any credit for getting into a favorable count (or debit for falling into a bad count) gets perfectly offset by the final result of the plate appearance.
   124. Kiko Sakata Posted: March 21, 2019 at 05:56 PM (#5824439)
There's no such thing as a sub-event level.


Fair enough. I was using his language. Pitch vs. plate appearance is more accurate. Certainly it would be ridiculous to try to give the pitcher credit for what the ball does for the first 30 feet and the catcher what the same pitch does over the next 30 feet.
   125. Esteban Rivera Posted: March 21, 2019 at 05:59 PM (#5824440)
I'm not sure I buy this - i.e., I think your argument about diving down to the sub-event has some merit. And I definitely don't buy the actual magnitude of numbers shown by Fangraphs (and, to be fair to them, by others, too; everybody who's tried to quantify this has ended up in the same general ballpark). But that's why we would treat catchers and hitters differently at the sub-event level.


I'm not really buying it that much either, i think they're giving too much credit for something that most of the time ends up being inconsequential in the final result of the plate appearance. I mentioned this earlier, but the only times I would think you could/should give value for pitch framing would be when there is a tangible final outcome from the pitch (which is when the pitch directly results in a strikeout or a walk due to the framing). All other pitches in the at bat are just noise in the lead up to the final outcome. This really feels to me like a 'Glenn Hubbard is the greatest second base fielder of all time' moment from when TPR was the stat du jour for player value, only for later advances/refinements showing that it was not really the case. There is value in pitch framing, but it doesn't make sense to me that it should be accounted and included for every pitch handled (since most of the time these framed pitches don't usually matter if the final event is not a strikeout or walk). Whatever effect it has will likely show up more in strikeouts and walks allowed than in runs allowed.
   126. Cleveland (need new name) fan Posted: March 21, 2019 at 06:02 PM (#5824441)
The thinking underlying this is so bizarre I'm not sure I could even begin to guess at it. Like, have you ever met a human being before? Grocery stores play slow music so people will buy more, and they do, but I'd guess ~0 of them think it through as "Hmm, they're playing Kenny G & Celine Dionne, I better put a few more bags of flour in the cart".


OK you don't have an answer so you attack the person.

Lets try it again, slowly. Umpires know that catchers are trying to manipulate them, they know the techniques that catchers are using to try to manipulate them, but they continue to be manipulated. Agreed, ignoring people trying to manipulate you can be difficult, but is there any evidence that the umpires are trying to call better balls and strikes and ignore pitch framing techniques?
   127. SoSH U at work Posted: March 21, 2019 at 06:31 PM (#5824444)
Lets try it again, slowly. Umpires know that catchers are trying to manipulate them, they know the techniques that catchers are using to try to manipulate them, but they continue to be manipulated. Agreed, ignoring people trying to manipulate you can be difficult, but is there any evidence that the umpires are trying to call better balls and strikes and ignore pitch framing techniques?


First of all, it's not about manipulation. Most of the attempts to manipulate (pulling the ball back over the plate after you caught it just aren't effective). It's simply about catching the ball. That's why the whole "framing" and "stealing" and "cheating" aren't very helpful to the discussion, because that's not what it's really about. It's about hitting the target and catching the ball quietly.

But let's try this.

As an ump, you're set up on the inside corner. The catcher is set up on the inside corner. The pitch fools him, and is instead outside. He gets a late read and has to lunge across the plate to reach it. As an ump, the movement of the catcher is going to make the pitch look like a ball, even if it catches the outside corner. In this case, the catcher obviously wasn't trying to manipulate you. He's just trying to catch the ball. Do you really find it difficult to believe that the home plate umpire, in this instance, could mistakenly judge this strike as a ball? I hope not, because it happens all the time. And if you do understand how that can happen, you ought to be able to understand that the reverse works as well. That if the pitcher happens to throw the pitch exactly where the catcher was set up, it's going to make the pitch look better. It's all about perception of an event in a fraction of a second.
   128. JAHV Posted: March 21, 2019 at 06:46 PM (#5824449)
Lets try it again, slowly. Umpires know that catchers are trying to manipulate them, they know the techniques that catchers are using to try to manipulate them, but they continue to be manipulated. Agreed, ignoring people trying to manipulate you can be difficult, but is there any evidence that the umpires are trying to call better balls and strikes and ignore pitch framing techniques?


I've only umpired lower levels, but your mind has to very quickly process something that has an infinite number of outcomes using a lot of visual data that you try to reduce into easily manageable information that your brain can process. "Did the pitch cross the plate in the batter's strike zone" gets broken down into a number of different questions, some of which should be irrelevant but which are part of your field of vision. The best umpires can filter out most of the irrelevant data, but they'll never filter out all of them.

How fast is the pitcher throwing? Was the pitch a breaking ball? How tall is the batter? These things all matter. But they get mixed up with other visual clues: Did the catcher have to move his glove significantly? Did the batter start to offer at the pitch? Has the pitcher been accurate this game? Has he been accurate in the past few batters? Those questions SHOULDN'T matter if each pitch is a discrete event that has to be judged solely on its merits. But it's hard to get rid of it all since these are things you see and your brain wants to take them into account. It's like false evidence in a criminal investigation.

You can add "Is the catcher trying to fool me with pitch framing?" into the mix, but that question should also be irrelevant to the actual point where the ball crossed the plate. Again, I've umpired only lower levels, so the pitch framing I've seen is very obvious when a catcher tries it, and I'm able to keep it from entering into my decision-making. But I'd imagine that if it did, that would make me more likely to call a ball than a strike, since now my brain is conditioning itself to assume that what looks like a borderline strike might just be a catcher trying to fool me. I wouldn't want that assumption to creep into my thought process.
   129. Never Give an Inge (Dave) Posted: March 21, 2019 at 07:15 PM (#5824454)

Exactly, and there's still the "sub-event" problem. Can a batter really "control" the caliber of pitches he sees? If not, why is he penalized if he sees better pitches? If a hitter sees pitches during the season that the league would hit .230/.310/.370 against and he hits .250/.330/.390, has he added value? If not, why not?

I actually think that this is something that we would want to account for if we could. Sabermetrics generally has assumed that things like difficulty of opposing pitchers evens out over the course of the season or career, but BPro used to track this information and some hitters definitely faced tougher opposing pitchers over the course of a season. If you could easily do something even more granular, at the pitch level, I don't see why you wouldn't.

When we talk about a player's Runs/Wins above Average/Replacement it should be against how an Average/Replacement level player would perform in the same context. So the difficulty of the pitches they face would be part of that. Certainly in terms of predicting the future, you'd want to know whether Bryce Harper had a down season last year because he happened to face tougher pitches than in prior seasons. (Then you'd want to look at whether those tougher pitches were due to random chance or whether that might persist in the future.)

It does for catchers, too. The "gained" strike is swallowed up by the final event with catchers just as with hitters. This reality is independent of the process of dividing up "credit."

To Kiko's point, whatever a hitter gains in value by hitting an 0-2 pitch, he lost earlier in the at-bat by falling behind 0-2. There's no other offensive player to apportion that value to.

Whereas a catcher might help gain two strikes through framing, but if the batter then gets a hit on the 0-2 pitch, that's arguably all on the pitcher/fielders. The value produced by the catcher via framing on the first two pitches was lost by the pitcher/fielders who allowed a hit on the 0-2 count.
   130. . Posted: March 21, 2019 at 07:30 PM (#5824459)
To Kiko's point, whatever a hitter gains in value by hitting an 0-2 pitch, he lost earlier in the at-bat by falling behind 0-2. There's no other offensive player to apportion that value to.


The second part is certainly true, but the first part isn't exactly true because the value is context-dependent and wouldn't necessarily add up to zero. I'd commend the PGA Tour's strokes gained idea (easily Googleable), as it has a lot of relevance to this discussion. One way to get a par on a hole is to hit every shot to the precise baseline, so your strokes gained is zero. Another is to his a great drive, a so-so second shot, a good putt, and then hole out. There, your strokes gained might not sum to zero. You can gain 0.11 strokes while making a par; you can gain 0 strokes while making a par. Similarly, if the first two strikes are baseline with your peers you haven't really lost any value and then you gain value by outperforming your peers by hitting the double on 0-2.

Another way to put it is that "fault" -- and therefore value lost -- isn't the same for every 0-2 situation.

   131. . Posted: March 21, 2019 at 07:40 PM (#5824462)
Strokes gained:

The Strokes Gained statistic shows how many strokes were gained from a particular shot type compared against the PGA Tour benchmark, based on distance to the home from the start of the shot. Positive numbers represent strokes lost and indicate better performance.

Here is a brief description from the founder of the statistic himself, Mark Broadie:

Here's a brief explanation of strokes gained: If a stroke starts on a tee where, according to historical data, the average score is four, and if it finishes at a position in the fairway where the average strokes to hole out is 2.8, then the tee shot has moved the ball 1.2 strokes closer to the hole with just one stroke. The single tee shot has gained 0.2 strokes compared to an average tee shot, so it has a "strokes gained" of 0.2. Strokes gained recognizes that sinking a 20-foot putt represents a better performance than sinking a three-foot putt, even though they both count as a single stroke on the scorecard. Strokes gained assigns a number to this intuition.
   132. Never Give an Inge (Dave) Posted: March 21, 2019 at 07:41 PM (#5824464)

The second part is certainly true, but the first part isn't exactly true because the value is context-dependent and wouldn't necessarily add up to zero.


Right, some pitches or at-bats are harder than others. Getting a hit in those situations should be considered more valuable.

But any additional difficulty of hitting a pitch in an 0-2 count versus hitting that same pitch in an 0-0 count is offset by the fact that it was the same hitter who dug himself into the 0-2 hole. (Unless, of course, there was a pinch hitter brought in mid-at-bat).

To use your golf analogy, if you hit par on a hole, your total "strokes gained" for that hole should be the same regardless of whether you hit a crappy tee shot and then made a great putt, if you hit a great tee shot but then only had to sink an easy putt at the end. (I assume this is how the stat works?). I think of an at-bat the same way I would think of a golf hole -- you can't create extra value overall by digging yourself into a difficult situation.
   133. . Posted: March 21, 2019 at 07:47 PM (#5824465)
But any additional difficulty of hitting a pitch in an 0-2 count versus hitting that same pitch in an 0-0 count is offset by the fact that it was the same hitter who dug himself into the 0-2 hole.


In part, but not entirely. 98 on the black, take; 98 on the black, take; double is a better AB than 90 down the middle, take; 90 down the middle, take; double.

I think of an at-bat the same way I would think of a golf hole


This is the right way to think about it. I'm not 100% sure of the answer to your earlier question, though I'm strongly leaning "no." You can have the same ultimate "event" on a hole -- the final score -- and yet have different total strokes gained. Similarly, it probably makes sense to do the same thing with a "final score" of "double."
   134. Kiko Sakata Posted: March 21, 2019 at 08:08 PM (#5824470)
I decided to try to dig a little more into Ryan Doumit's 2008 season to try to get a handle on whether it's plausible that he cost the Pirates 57.8 runs, as Fangraphs says.

In Baseball-Reference's pitching splits (team or player), one of the splits is by catcher.

As noted earlier, the Pirates had a team ERA of 5.08, which was terrible in 2008 (ERA+ of 82). They only had one starting pitcher who was any good at all (ERA+ higher than 87): Paul Maholm, who had a 3.71 ERA. I checked to make sure the issue wasn't that Doumit was Maholm's personal catcher. Maholm pitched 206.1 innings, of which Doumit caught 121.1 - 58.8%. Doumit caught 62.5% of the Pirates' total innings in 2008, so nothing really to see there. Incidentally, Maholm had an ERA of 3.12 pitching to Doumit (2.72 pitching to Raul Chavez in 6 games, 6.43 pitching to Ronny Paulino in 7 games).

I didn't bother to look at any other Pirate pitchers: I'm not writing a thesis or getting paid for this after all.

Here, then, are a few numbers for Doumit vs. other Pirates catchers that year.

.          IP    R     PA      H   HR    W    K  HBP  GDP
Doumit    909  546  4
,080  1,011  105  435  581   27  102
Others    546  338  2
,448    620   71  222  382   22   55
Others
(*) 909  563  4,076  1,032  118  370  636   37   92

(*) Adjusted to match Doumit's IP 


So, as was pointed out on the first page: the Pirates didn't allow more runs when Doumit was catching - and changing from ER to R doesn't change that - they actually allowed a bit fewer runs (17 in 909 innings or 0.17 runs per 9 innings).

But Doumit did allow more walks - about 65 more over Doumit's 909 innings - and fewer strikeouts - by about 55. Pirates' pitchers hit 10 fewer batters with Doumit catching. I don't know what that has to do with pitch framing but if you add walks and HBP, you get the nice result that Pirates pitchers allowed 55 more batters to reach first base and struck out 55 fewer batters with Doumit catching than with their other catchers.

So, certainly, the K and W numbers look like you'd expect for a bad pitch-framer. But how much does that translate into in terms of runs? One of the articles linked to Fangraphs' explanation of their pitch framing numbers said that a walk was worth 0.31 runs and a strikeout was worth -0.28 runs (to the offensive team, obviously). Those seem reasonable enough, so let's use them. That means turning a strikeout into a walk costs the defensive team 0.59 runs. Multiply by 55 and, voila, 32.5 runs due to Doumit's pitch framing.

Assuming, of course, that all of the difference in walks and strikeouts should be blamed on Doumit. Oh, and Fangraphs said that the other Pirates' catchers were +6.1 runs, which blows up to +10.2 over Doumit's innings. So, Doumit isn't 32.5 runs worse than average; he's 32.5 runs worse than the other Pirates' catchers, who are 10.2 runs above average. So, Doumit is only 22.3 runs worse than average due to pitch framing.

Which is a pretty far cry from 57.8.

Ah, but what about bad frames that don't lead to walks or cost strikeouts but merely lead to worse counts for his pitchers?

Well, I'm looking at the data and I'll be damned if I see the actual damage there. Doumit allowed 21 fewer hits than the other catchers, including 13 fewer home runs. He got 10 more GDP - as you'd expect given the additional 55 runners he put on first base. Doumit got 55 fewer strikeouts and 65 more walks but that only led to 4 extra plate appearances, due to fewer hits, fewer hit batters, and more double plays (and I presume a few extra outs on the bases, because the numbers don't precisely line up otherwise).

I don't know. I'm not seeing 57.8 runs of difference between the first and third line in the above tables. Honestly, I'm not completely convinced that there's any meaningful difference in runs between those two lines.
   135. Sunday silence Posted: March 21, 2019 at 08:16 PM (#5824471)
Imagine getting touched up for 5 runs on 6 hits and 3 walks in 6 inn. You'd feel pretty bad wouldnt you?

but wait a minute! It was with that putz Dumit behind the plate. Hell that's the equivalent of throwing a 2 hit shutout with Yadier behind the plate!

its a win!
   136. Cleveland (need new name) fan Posted: March 21, 2019 at 08:46 PM (#5824475)
SOSH and JAHV, thank you for clearly explaining the issues to this non-umpire.

The catcher is set up on the inside corner. The pitch fools him, and is instead outside. He gets a late read and has to lunge across the plate to reach it. As an ump, the movement of the catcher is going to make the pitch look like a ball, even if it catches the outside corner. In this case, the catcher obviously wasn't trying to manipulate you. He's just trying to catch the ball. Do you really find it difficult to believe that the home plate umpire, in this instance, could mistakenly judge this strike as a ball?


I'm really not saying this very well, so let me paraphrase your and JAHV answers to see if I understand the umpires position. The ball and strike call takes place a couple of feet in front of the catcher, so anything he does should be irrelevant. However, because of the speed of the play, the umpire can't make his decision fast enough, so extraneous factors, like the catcher's actions come into play and the call becomes a gestalt of the entire play, even the irrelevant parts. The human mind then can't separate the important part (where the ball crosses the plate) from the unimportant parts like how the catcher caught the ball in the split second needed to make the decision. Essentially an optical illusion. Does this capture your thoughts correctly?
   137. SoSH U at work Posted: March 21, 2019 at 10:11 PM (#5824480)
I'm really not saying this very well, so let me paraphrase your and JAHV answers to see if I understand the umpires position. The ball and strike call takes place a couple of feet in front of the catcher, so anything he does should be irrelevant. However, because of the speed of the play, the umpire can't make his decision fast enough, so extraneous factors, like the catcher's actions come into play and the call becomes a gestalt of the entire play, even the irrelevant parts. The human mind then can't separate the important part (where the ball crosses the plate) from the unimportant parts like how the catcher caught the ball in the split second needed to make the decision. Essentially an optical illusion. Does this capture your thoughts correctly?


That's a pretty reasonable way of putting it.
   138. villageidiom Posted: March 22, 2019 at 01:02 AM (#5824496)
To Kiko's point, whatever a hitter gains in value by hitting an 0-2 pitch, he lost earlier in the at-bat by falling behind 0-2. There's no other offensive player to apportion that value to.

Whereas a catcher might help gain two strikes through framing, but if the batter then gets a hit on the 0-2 pitch, that's arguably all on the pitcher/fielders. The value produced by the catcher via framing on the first two pitches was lost by the pitcher/fielders who allowed a hit on the 0-2 count.
Today we don't account for the count when measuring defense or pitching. If we're saying a double on a 0-2 count is more costly than a double on a 2-0 count, and we're going to penalize the fielders for having undone the value the catcher achieved, that's one thing. But we don't do that now for fielders, regarding pitchers achieving that count without framing. We don't for any reason penalize or credit defenders for outcomes relative to expected outcome for the given count. We do it for outcomes relative to expected outcome. A ball hit with this velocity has an expected run value of +0.5 for the offense, they catch it and produce a -0.3 outcome, the fielder is credited for saving 0.8 runs. We don't start with an expectation from an 0-2 count because that's all meaningless for the defender once the ball is actually in play. And we don't credit or debit pitchers based on the counts they achieve; we credit or debit them on defense-independent stats and ignore the BIP.

If one wishes to argue that the count influenced the fielder's positioning or something like that, great, but we shouldn't do it for catcher framing without also doing it for pitching. If a 2-0 count vs. an 0-2 count should influence positional defensive metrics (because each has a different run expectation) then it shouldn't matter to the measuring of positional defense whether those counts were produced by the catcher or the pitcher. Just like the batter starts at 0-0 and we don't look at his outcome value except in relation to that, the defender starts at whatever count produced the BIP, and the pitcher starts at 0-0 and ends at the outcome, for those events where the defense doesn't get involved. If we are to account in pitching or defense for the creation of the count prior to the BIP, but only for the catcher's contribution to it, that makes no sense.

Again I'm not saying we shouldn't account for it. I'm just saying this accounting is incompatible with the rest of the accounting we do.
   139. JAHV Posted: March 22, 2019 at 02:50 AM (#5824498)
I'm really not saying this very well, so let me paraphrase your and JAHV answers to see if I understand the umpires position. The ball and strike call takes place a couple of feet in front of the catcher, so anything he does should be irrelevant. However, because of the speed of the play, the umpire can't make his decision fast enough, so extraneous factors, like the catcher's actions come into play and the call becomes a gestalt of the entire play, even the irrelevant parts. The human mind then can't separate the important part (where the ball crosses the plate) from the unimportant parts like how the catcher caught the ball in the split second needed to make the decision. Essentially an optical illusion. Does this capture your thoughts correctly?


I think that gets to the gist of it, yes. The speed at which you have to make the call is crucial. I think the brain develops some autopilot tendencies from the repetition of seeing a certain type of play and associating it with a certain result. To use SoSH's example, after seeing a catcher lunge to catch a ball 10,000 times, and having called 9,999 of those balls, your brain does a quick, subconscious calculation: "That's probably a ball." And in that split second, it's going to take some serious override from your conscious thought to determine that even though the catcher had to jump to catch it, it was a strike.

Similarly, when you see a catcher set up on a corner of the plate and he catches a pitch with very little to no movement of his glove, your brain goes into autopilot and wants to call it a strike, because it usually is a strike. The best catchers know how to make the umpires see what they want them to see, so in that sense, it is an optical illusion.
   140. DL from MN Posted: March 22, 2019 at 09:32 AM (#5824520)
I thought a step further about this. Fangraphs doesn't give credit to the pitcher for any ball in play but now they're giving credit to the catcher on those same plays. I don't understand how someone can use FIP for their pitcher accounting and give credit to catcher framing for at-bats where the ball is put in play. I think there has to be some consistency here.

Plate appearances that end with balls in play are only debited/credited to the fielders near where the ball is hit (which usually isn't the catcher). Strikeouts and walks are only debited/credited to the pitchers and catchers. Catchers only get partial credit and that credit becomes higher if there are "frameable" pitches near the edge of the strikezone during the plate appearance. I still couldn't see how the catcher gets more than 50% of the value since the pitcher has to hit the spot. This gives catchers more credit than they were getting for strikeouts and walks (previously zero) but is consistent with the whole philosophy of FIP that pitchers don't have control over what happens to a ball that is put in play.

Getting ahead in the count makes it much more likely that the batter will have to swing the bat. Getting a batter to swing is generally a good thing because those plate appearances tend to end in outs. However, what happens after the ball hits the bat is not something the pitcher (or catcher) can control with any more consistency than any other pitcher (or catcher) in the big leagues.
   141. Howie Menckel Posted: March 22, 2019 at 09:48 AM (#5824523)
if I didn't know better, I would think that hitting stats are more reliable than fielding ones and - work with me here - maybe WAR isn't quite the "end of discussion" stat that we are supposed to think it is when comparing players.

I will now leave my door unlocked, to aid the authorities in rounding me up and sending me to the death chamber. It's ok, I had a good run.
   142. . Posted: March 22, 2019 at 09:53 AM (#5824525)
This gives catchers more credit than they were getting for strikeouts and walks (previously zero) but is consistent with the whole philosophy of FIP that pitchers don't have control over what happens to a ball that is put in play.


For my money, the entire "control" idea is faulty. In part, philosophically -- it's far too Randian; in part because it really isn't accurate. The only thing a hitter really controls is the coordinates at which he contacts and launches the ball and even that isn't granular enough because he doesn't control the quality of pitches he receives.

So my solution is to kind of ignore the whole concept, recommit to the idea that baseball is a team sport with a lot of moving parts, recommit to the idea that there's nothing untoward about "good luck," and to simply revert for the most part to the traditional ex post accounting units that describe what actually happened in actual games. Why is what might have happened in a different context even important?

I hit a great 10 foot putt that should have gone in, but it hit a spike mark three feet from the hole and so it lipped out. How should that event be interpreted?
   143. DL from MN Posted: March 22, 2019 at 09:56 AM (#5824528)
I decided to try to dig a little more into Ryan Doumit's 2008 season to try to get a handle on whether it's plausible that he cost the Pirates 57.8 runs, as Fangraphs says.


Thanks for doing this. It was enlightening.
   144. jmurph Posted: March 22, 2019 at 10:29 AM (#5824539)
I decided to try to dig a little more into Ryan Doumit's 2008 season to try to get a handle on whether it's plausible that he cost the Pirates 57.8 runs, as Fangraphs says.

Again if I am to understand them correctly, in the 2008 season Doumit cost his team 5.8 wins through his poor framing alone. Jose Reyes had a 5.8 WAR year that season, good for 18th on the WAR leaderboard. Doumit cost his team the same amount of value, exclusively through his receiving of pitches, as that produced by the 18th overall best season in MLB that year.

(If I am misstating any of this in my attempt at snark please correct me, it's totally possible I am.)
   145. . Posted: March 22, 2019 at 10:34 AM (#5824542)
Again if I am to understand them correctly, in the 2008 season Doumit cost his team 5.8 wins through his poor framing alone. Jose Reyes had a 5.8 WAR year that season, good for 18th on the WAR leaderboard. Doumit cost his team the same amount of value, exclusively through his receiving of pitches, as that produced by the 18th overall best season in MLB that year.


That fact alone should prompt an immediate "Return to Sender."
   146. Rally Posted: March 22, 2019 at 10:36 AM (#5824544)
Interesting that the Pirates didn't actually give up more runs when Noumitt was catching, but not really any more meaningful than seeing that the Angels in 2018 went 69-71 in games Mike Trout played, and 11-11 in games he did not play in. Just a fluke. I don't think many would take that to mean that having or not having Trout makes no difference to the Angels' record. Certainly Arte Moreno doesn't buy that and has backed up that belief with 430 million dollars, though of course he's been wrong before on big money bets.

If we found that over multiple years and multiple catchers that the framing numbers did not correspond to real runs, that would be an indication that the framing results were not meaningful.

Way back when people like Mike Fast and Max Marchi were in the public domain instead of working for teams, MGL took their lists of good and bad framing catchers and tried to see if it matched up to actual baseball outcomes with a WOWY analysis. He found results consistent with framing being a real and useful skill. Here's a link:

http://www.insidethebook.com/ee/index.php/site/comments/testing_catcher_framing_numbers/
   147. Karl from NY Posted: March 22, 2019 at 10:36 AM (#5824545)
Certainly it would be ridiculous to try to give the pitcher credit for what the ball does for the first 30 feet and the catcher what the same pitch does over the next 30 feet.

Now I have the image of a Molina's gravitational field bending a ball's trajectory more as it approaches the plate. That's a component that WAR would want to account for, right?
   148. Pat Rapper's Delight (as quoted on MLB Network) Posted: March 22, 2019 at 10:40 AM (#5824547)
They'll have to round me up with you, Howie. I usually lurk only in the pitch framing threads because I've never really dug into the details behind the calculations. Perhaps it's my Old Man Yells At Cloud moment, but the idea that -- at the major league level especially -- the difference between the best and worst catchers due to pitch framing alone has an All-Star/borderline MVP level of WAR difference pegs my internal BS meter so strongly that I'm tempted to reject all of it out of hand as pure nonsense.
   149. Karl from NY Posted: March 22, 2019 at 10:42 AM (#5824548)
I decided to try to dig a little more into Ryan Doumit's 2008 season to try to get a handle on whether it's plausible that he cost the Pirates 57.8 runs, as Fangraphs says.

So, Doumit isn't 32.5 runs worse than average; he's 32.5 runs worse than the other Pirates' catchers, who are 10.2 runs above average. So, Doumit is only 22.3 runs worse than average due to pitch framing.

Is the 57.8 comparing to replacement level (presumably what WAR would be doing), while the 22.3 number is comparing to average?
   150. What did Billy Ripken have against ElRoy Face? Posted: March 22, 2019 at 10:44 AM (#5824549)
Now I have the image of a Molina's gravitational field bending a ball's trajectory more as it approaches the plate. That's a component that WAR would want to account for, right?
"Hello, Bartolo? It's your agent. Still looking to make a comeback? I've got an idea..."
   151. DavidFoss Posted: March 22, 2019 at 10:47 AM (#5824551)
Is the 57.8 comparing to replacement level (presumably what WAR would be doing), while the 22.3 number is comparing to average?

In modern WAR calculations, average fielding *is* replacement. The idea being that AAA is filled with good-field-no-hit types.

BaseballProspectus experimented using a lower replacement level for fielding in the early 00s and their "WARP" values were too defense-centric for most people's tastes.
   152. . Posted: March 22, 2019 at 10:55 AM (#5824553)
Interesting that the Pirates didn't actually give up more runs when Noumitt was catching, but not really any more meaningful than seeing that the Angels in 2018 went 69-71 in games Mike Trout played, and 11-11 in games he did not play in.


That seems a bit strong. If Doumit was really that bad at pitch framing, it makes perfect intuitive sense that we'd be able to find some evidence that his badness translates to more runs scored by the other team.
   153. SoSH U at work Posted: March 22, 2019 at 11:03 AM (#5824557)
Interesting that the Pirates didn't actually give up more runs when Noumitt was catching, but not really any more meaningful than seeing that the Angels in 2018 went 69-71 in games Mike Trout played, and 11-11 in games he did not play in.


That seems one step removed from the comparison at hand to me. How the Pirates fared in the games Ryan Doumit caught that year is more comparable to the Trout example. Or conversely, that Angels' center fielders scored the same number of runs with or without Trout in the lineup is akin to the Pirates claim.
   154. Kiko Sakata Posted: March 22, 2019 at 11:17 AM (#5824565)
Way back when people like Mike Fast and Max Marchi were in the public domain instead of working for teams, MGL took their lists of good and bad framing catchers and tried to see if it matched up to actual baseball outcomes with a WOWY analysis. He found results consistent with framing being a real and useful skill. Here's a link:

http://www.insidethebook.com/ee/index.php/site/comments/testing_catcher_framing_numbers/


Thanks for sharing that link, Rally.
   155. Bote Man Posted: March 22, 2019 at 11:22 AM (#5824568)
Russell Roberts @EconTalker
Data-based evidence always helps us make better decisions except when it doesn't. And I don't think our brains are that good at knowing which is which. This is the challenge of modernity in a world of growing data.

Bill James Online @billjamesonline
Replying to @EconTalker
That's right. One can drown in data. Those who make a living by creating data have no incentive to say so, but at the moment, we can create data much faster than we can understand it. We're getting buried in data like that criminal in "Witness" getting buried in a grain silo.
   156. . Posted: March 22, 2019 at 11:23 AM (#5824570)
Those who make a living by creating data have no incentive to say so


See 116 ....
   157. Tom Nawrocki Posted: March 22, 2019 at 11:27 AM (#5824571)
Interesting that the Pirates didn't actually give up more runs when Noumitt was catching, but not really any more meaningful than seeing that the Angels in 2018 went 69-71 in games Mike Trout played, and 11-11 in games he did not play in. Just a fluke.


It's really not all that comparable. For one thing, the level of competition in 22 missed games is not going to be the same as the competition over the course of the season. With Trout out of the lineup last year, the Angels swept series against the shitty Tigers and the shitty Padres, and still managed to go 1-3 against the shitty Rangers. Out of those 22 games he did not play in, 10 were against teams that lost 95 or more games. The Trout-less Angels went 7-3 against teams that lost 95 or more games, and 4-8 against everyone else.

For another thing, we have a reasonably good idea of how many runs Mike Trout provides the Angels on offense, and a shakier but still reasonable idea of how many he prevents on defense. We have to take it on faith that Ryan Doumit cost his team 5.8 wins or 58 runs, but we know for a fact that the 2008 Pirates did not allow more runs with him in the lineup than they did without him in the lineup.
   158. Karl from NY Posted: March 22, 2019 at 11:57 AM (#5824576)
W/L record with a player is a stupid comparison here. Why would you deliberately introduce a ton of noise that overwhelms the individual contribution? Even runs is too coarse to want to use when we've got data for every component per pitch and PA.
   159. snapper (history's 42nd greatest monster) Posted: March 22, 2019 at 12:00 PM (#5824578)
Even runs is too coarse to want to use when we've got data for every component per pitch and PA.

If you can't show a direct link to runs, it's not clear why we care about the components.

If, over the course of a large sample, the Angels didn't score more runs with Trout than without, I would definitely question the methodolgy that says he's the best player in MLB.
   160. . Posted: March 22, 2019 at 12:54 PM (#5824609)
Why would you deliberately introduce a ton of noise that overwhelms the individual contribution?


Sure, but the flip side question would be: Why are people so concerned with the "individual contribution"?

Fans don't really act that way, in reality -- right? Who went home after Bobby Thomsen's homer and said things like, "What's the big deal, he was lucky that there were two men on base when he came up he didn't have anything to do with that it was outside his control"?

Why would anyone write off the fact that there were two men on base rather than none as mere "noise"?
   161. David Nieporent (now, with children) Posted: March 22, 2019 at 01:01 PM (#5824615)
Way back when people like Mike Fast and Max Marchi were in the public domain instead of working for teams, MGL took their lists of good and bad framing catchers and tried to see if it matched up to actual baseball outcomes with a WOWY analysis. He found results consistent with framing being a real and useful skill. Here's a link:
I don't think it's particularly controversial to argue that it's a real and useful skill. (Though of course some may argue that we should have automated balls and strikes so that it won't be.) The controversy is the magnitude of the effect. And unless I'm misreading that link you provide, it doesn't remotely support the idea of an effect the size that Fangraphs is suggesting.
   162. PreservedFish Posted: March 22, 2019 at 01:10 PM (#5824621)
Great conversation here. I share everyone else's skepticism.

But I do think a few things are getting downplayed here. One, Doumit's -5.8 WAR season looks like an epic and unaccountable fluke. There are only three more seasons at -3 WAR. We should use that as our realistic upper bound for this phenomenon. Second, while the -5.8 wins sounds insane, remember, this is 900 innings. If I did the math right, we're talking about something like 0.50 points of ERA. That's a lot, but when it's happening, it's too small to notice and could easily get washed out by other factors, like luck, and virtually disappear. The other really crappy framers, like Laird and Marsten, were a negative drag of 0.25 points of ERA ... totally unnoticeable over anything but a multiyear sample.

I don't think we can just dismiss these numbers. I think they're close enough to plausible that we need to wrestle with them.

   163. villageidiom Posted: March 22, 2019 at 01:21 PM (#5824624)
I don't think we can just dismiss these numbers. I think they're close enough to plausible that we need to wrestle with them.
In terms of ranking catcher performance, they seem to work very well. In terms of magnitude of value, compared with non-framing activities like pitching and hitting and defense, they don't. I keep using the word incompatible* not because the numbers are wrong, but that the numbers are considering something (achieving a different count during a PA) as being of value for catchers, but not for pitcher (or hitter or defense).

It's like deciding who wins a baseball game in which all the visiting team's runs count but the home team's runs only count if they came by HR. They're both a tally of runs! They're on the same basis! Uh, no.

* It means what I think it means.
   164. villageidiom Posted: March 22, 2019 at 01:23 PM (#5824629)
I mean, we had people raising this issue on this site like 4-5 years ago. At that time someone (not me) said something like "yeah the magnitude might be off, but that's only a problem if you try to fold it into WAR or something like that." This isn't a new thing, except that someone went and tried to do just that.

EDIT: Found it. From 4 years ago:
But credit is given on stolen/lost strikes whether the PA results in a K or not. It's like WPA taken to its exteme and credit is given for the change in expected outcome on every pitch but that credit/debit is being given to only one player on that basis -- i.e. the pitcher/defense gets the same credit/debit for a hit on 2-1 as on 1-2. On average, that probably works out OK but it's a funny way to do things.

That's an issue to address if framing is being worked into a WAR system, not an issue with the framing numbers themselves. I'd think that when looking at pitchers, you'd adjust for catcher framing similarly to how bWAR adjusts for team defense.
   165. . Posted: March 22, 2019 at 01:33 PM (#5824636)
We've had similar discussions regarding the defensive "scaling." Units of defense and units of offense aren't the same either. Hitters get full credit for mashing BP caliber fastballs down the middle; defenders don't get full credit for making a play on the defensive analogue. (And that doesn't even touch on the underlying assumption that the replacement bin is filled with major league caliber gloves.)

   166. PreservedFish Posted: March 22, 2019 at 01:50 PM (#5824643)
#134 is an excellent comment. The BB/K disparity between Doumit and his backups is big and legit, although of course it's impossible to know how much of it was his "fault."

If Kiko properly found Doumit's effect on BBs and Ks, then the remaining -35 runs must come from batted ball events. That is, because the batters are hitting in better counts, they're hitting the ball harder.

Ryan Doumit was behind the plate for something like 3000 balls in play (including homeruns). We're trying to find 35 runs in those 3000 events. So what we're asking is if it's plausible that Doumit made the average hitter ~0.012 runs better per ball in play, and thus, if the other worst catchers like Marston and Laird made the average hitter ~0.006 runs better, assuming they have a similar spread in on-contact damage vs BB/K damage.
   167. Hysterical & Useless Posted: March 22, 2019 at 01:52 PM (#5824646)
No reason to credit the catcher for anything -- whether shared or otherwise


The reason to credit the catcher is for answering the question, Which catcher would you rather have (other things being equal), the one who puts your pitcher in a better position to succeed or the one who...doesn't?

Note, that doesn't mean that this data belongs in WAR, or that the numbers presented are accurate or even plausible. Just that, given that we know there truly are "pitcher's counts" vs "hitter's counts," the catcher who does a better job of helping his pitcher get to favorable counts is in fact doing a better job. So if it can be measured it likely will be.
   168. PreservedFish Posted: March 22, 2019 at 01:56 PM (#5824647)
In terms of ranking catcher performance, they seem to work very well. In terms of magnitude of value, compared with non-framing activities like pitching and hitting and defense, they don't.


I think it's tough to wrap your mind around the frequency of these events. Doumit caught 900 innings, which is far more than even Old Hoss Radbourn (19 WAR in 1884!) ever approached.

If Doumit is the shittiest catcher ever, and tons of hitters are sitting in better counts than they would otherwise, is it insane to suggest that about 1 out of every 100 hitters might create an extra run because of it?
   169. Tom Nawrocki Posted: March 22, 2019 at 01:57 PM (#5824648)
If I did the math right, we're talking about something like 0.50 points of ERA. That's a lot, but when it's happening, it's too small to notice and could easily get washed out by other factors, like luck, and virtually disappear. The other really crappy framers, like Laird and Marsten, were a negative drag of 0.25 points of ERA ... totally unnoticeable over anything but a multiyear sample.


Again, I'm not cherry-picking these examples. I'm looking at what Fangraphs says are the very worst catcher-framing seasons ever. Let's do some more:

Gerald Laird, 2009: 4.18 ERA with him behind the plate, 4.29 without him
Carlos Santana, 2011: 4.19 ERA with him behind the plate, 4.23 without him
Carlos Santana, 2012: 4.68 with him behind the plate, 4.78 without him

I've been through the Bottom Five all-time catcher framing seasons, and none of them dragged down their teams' ERA by 0.50 points, or even 0.25 points. It's not just that some of these players don't show as damaging their team's ERA. None of them show as damaging their team's ERA.
   170. PreservedFish Posted: March 22, 2019 at 02:02 PM (#5824653)
I'm aware, Tom. It's strange.
   171. Karl from NY Posted: March 22, 2019 at 02:19 PM (#5824670)
I've been through the Bottom Five all-time catcher framing seasons, and none of them dragged down their teams' ERA by 0.50 points, or even 0.25 points.

Do they drag down the component stats? Could try each of OBP, SLG, K%, BB%, HR%, BABIP, and see if the catcher framing comes out in any of those.
   172. Kiko Sakata Posted: March 22, 2019 at 02:20 PM (#5824672)
If Doumit is the shittiest catcher ever, and tons of hitters are sitting in better counts than they would otherwise, is it insane to suggest that about 1 out of every 100 hitters might create an extra run because of it?


I honestly don't know. But one thing I found looking around at some numbers is that the overwhelming difference in hitting by count shows up in walks and strikeouts. Here are the numbers for the 2008 NL (Ryan Doumit's league).

.              PA (less SH)       AB        H       TB   HBP       W        K   BAvg    OBP    SLG    OPS
After 1
-0 count      41,056   33,969    9,349   15,225   240   6,558    5,898   .275   .393   .448   .841
After 0
-1 count      46,675   43,700   10,027   15,370   470   2,258   12,152   .229   .273   .352   .625 


That's an OPS difference of .217, which is huge. Doing a simple runs-created formula (times on base * total bases divided by AB), the difference is 0.080 runs per PA. But by far the biggest differences between those two lines are the walks and the strikeouts. After a 1-0 count, batters draw three times as many walks - because they're one-quarter closer to a walk and because, I suspect, there's a selection bias that more walk-prone pitchers are more prone to falling behind 1-0. After an 0-1 count, hitters are more than twice as likely to strike out, for presumably the same reasons - better pitchers start 0-1 and you're one-third of the way to a strikeout.

Here, then, is what the lines look like if you remove walks and strikeouts. That is, I already explicitly accounted for the walks and strikeouts in #134; how much difference does a shitty frame on the first pitch of the plate appearance make in plate appearances that don't end in a walk or a K?

.              PA (less SH)       AB        H       TB   HBP       W        K   BAvg    OBP    SLG    OPS
After 1
-0 count      28,600   28,071    9,349   15,225   240       0        0   .333   .335   .542   .878
After 0
-1 count      32,265   31,548   10,027   15,370   470       0        0   .318   .325   .487   .813 


Now, the difference in OPS is only .065 and the difference in RC/pa is down to .023. Basically 70% of the difference between batting 0-1 vs. 1-0 is because of strikeouts and walks. Now, .023 isn't nothing, of course. But if #166 has the magnitude right - that Doumit's framing number implies a gap of .012 in runs/PA on balls in play, doesn't that imply that Doumit was screwing up the framing on a pitch every other plate appearance (.012 is about half of .023)? That can't possibly be true, can it?
   173. Never Give an Inge (Dave) Posted: March 22, 2019 at 02:21 PM (#5824675)

Interesting that the Pirates didn't actually give up more runs when Noumitt was catching, but not really any more meaningful than seeing that the Angels in 2018 went 69-71 in games Mike Trout played, and 11-11 in games he did not play in. Just a fluke. I don't think many would take that to mean that having or not having Trout makes no difference to the Angels' record.

Exactly. It's possible that the pitchers happened to throw better on days that Doumit caught, but his framing outweighed that, resulting in an identical catcher ERAs. Or maybe his superior game calling made up for a lot of his poor framing. I have no idea, but the point is that the type of analysis Fangraphs is doing is supposed to get us beyond just comparing cERAs. There is certainly reason to be skeptical of new stats like this and I would heavily regress the results until more work can be done, but I wouldn't immediately dismiss it because it doesn't align with blunter tools.
   174. snapper (history's 42nd greatest monster) Posted: March 22, 2019 at 02:25 PM (#5824676)
I honestly don't know. But one thing I found looking around at some numbers is that the overwhelming difference in hitting by count shows up in walks and strikeouts. Here are the numbers for the 2008 NL (Ryan Doumit's league).

But you can't use these raw numbers. The population of 1-0 counts is going to be heavily skewed to bad pitchers and good hitters, and the 0-1 count the other way. For any individual hitter-pitcher combo, the gap between 1-0 and 0-1 is going to be a lot smaller.

If the ump is at all aware that he might have been fooled by framing, the gap will be smaller still, as you can expect a bunch of makeup calls.
   175. Never Give an Inge (Dave) Posted: March 22, 2019 at 02:25 PM (#5824678)

I hadn't seen #169 before I posted #173. That is certainly more reason to be skeptical of the numbers. Hopefully Fangraphs publishes more follow-up here to help us understand the results.
   176. . Posted: March 22, 2019 at 02:37 PM (#5824682)
Ryan Doumit was behind the plate for something like 3000 balls in play (including homeruns). We're trying to find 35 runs in those 3000 events. So what we're asking is if it's plausible that Doumit made the average hitter ~0.012 runs better per ball in play, and thus, if the other worst catchers like Marston and Laird made the average hitter ~0.006 runs better, assuming they have a similar spread in on-contact damage vs BB/K damage.


I thought the relevant effect was on the 19,500 pitches (150 pitches*130 games), Doumit caught. According to the article Inge/Dave posted, a "stolen" strike is worth .161 runs, meaning that to get to even 30 runs, a catcher would have to "steal" 181 strikes above baseline. Given the number of pitches that are either (1) hit; (2) fouled; (3) clearly strikes; or (4) clearly balls ... that number seems very high. We're really talking about massive over or underperformance on taken pitches at or very close to the black.

Doumit's -5.8 WAR would mean he turned 360 baseline strikes into balls. That seems absurdly high, but that's only intuition speaking. (And to a degree the article, which found only a two run difference in the detailed scatter charts between Johjima, purportedly dreadful at framing, and Rob Johnson.)

   177. Never Give an Inge (Dave) Posted: March 22, 2019 at 02:51 PM (#5824690)

Doumit's -5.8 WAR would mean he turned 360 baseline strikes into balls. That seems absurdly high, but that's only intuition speaking. (And to a degree the article, which found only a two run difference in the detailed scatter charts between Uehara, purportedly dreadful at framing, and Rob Johnson.)

For what it's worth, the Fangraphs framing stats do agree with the article with respect to Johjima (not Uehara) and Johnson -- they were both bad.
   178. What did Billy Ripken have against ElRoy Face? Posted: March 22, 2019 at 02:53 PM (#5824691)
Sure, but the flip side question would be: Why are people so concerned with the "individual contribution"?

Fans don't really act that way, in reality -- right? Who went home after Bobby Thomsen's homer and said things like, "What's the big deal, he was lucky that there were two men on base when he came up he didn't have anything to do with that it was outside his control"?

Why would anyone write off the fact that there were two men on base rather than none as mere "noise"?
Some fans want to analyze and understand the game and players' abilities, including how players are likely to perform in the future or in different contexts. Other fans just want to cheer for what happens. Both ways are fine.
   179. . Posted: March 22, 2019 at 02:55 PM (#5824692)
Doumit also only caught 106 games in 2008, making his alleged WAR/162 games from bad framing: -8.8.
   180. . Posted: March 22, 2019 at 02:59 PM (#5824696)
Some fans want to analyze and understand the game and players' abilities, including how players are likely to perform in the future or in different contexts.


Do they though? Who wants to analyze how Bobby Thomsen would have performed if there had been no runners on base? I get the theoretical idea entirely; the issue though is that it really isn't followed through on to any significant degree.

   181. Never Give an Inge (Dave) Posted: March 22, 2019 at 03:11 PM (#5824698)

Do they though? Who wants to analyze how Bobby Thomsen would have performed if there had been no runners on base? I get the theoretical idea entirely; the issue though is that it really isn't followed through on to any significant degree.

Fans love to make predictions and form opinions about the moves that teams make. And a lot of them like playing fantasy baseball, even if you and I may not.

If your real team or the team that one of your fantasy starters was on was considering trading for Ryan Doumit, you might want to know what effect that would have on their ERA.
   182. What did Billy Ripken have against ElRoy Face? Posted: March 22, 2019 at 03:15 PM (#5824700)
Who wants to analyze how Bobby Thomsen would have performed if there had been no runners on base?
No, they want to know the answer to the very basic question of, How good was Bobby Thomson (or whatever other player) relative to his peers, or relative to other players? To really answer that question, which is a natural one, you have to be able to isolate ability from context, to the best of your ability. Come on, at some level you have to know this.
   183. DL from MN Posted: March 22, 2019 at 03:37 PM (#5824707)
Thanks again Kiko - 172 is pretty consistent with xFIP - performance on balls in play is pretty close for MLB pitchers. If we can't tell the difference between pitchers on balls in play how the hell are we supposed to tell the difference between catchers?

It would also mean framing matters a lot less when strikeouts and walks are low. As strikeouts and walks have trended to new highs then catcher framing becomes more significant.
   184. . Posted: March 22, 2019 at 05:23 PM (#5824743)
Fans love to make predictions and form opinions about the moves that teams make. And a lot of them like playing fantasy baseball, even if you and I may not.


I love to do that, too. I'm just not sure the added granularity has added much to the toolbox in that area. And to the extent it's actually caused overshoots like the pitch framing stuff, it's counterproductive.

Can I look at a trade or a team outlook in 2019 and predict it any better than in 1984? Probably not. There's a difference between information and knowledge.

No, they want to know the answer to the very basic question of, How good was Bobby Thomson (or whatever other player) relative to his peers, or relative to other players? To really answer that question, which is a natural one, you have to be able to isolate ability from context, to the best of your ability. Come on, at some level you have to know this.


I do know that, but your question is an entirely different one than the one I asked. In terms of list rankings versus peers, that's the kind of thing where the data has driven the desire for lists more than anything else. The desire for lists isn't really *that* organically natural. I do agree that it's somewhat natural.
Page 2 of 2 pages  < 1 2

You must be Registered and Logged In to post comments.

 

 

<< Back to main

News

All News | Prime News

Old-School Newsstand


BBTF Partner

Dynasty League Baseball

Support BBTF

donate

Thanks to
Andere Richtingen
for his generous support.

Bookmarks

You must be logged in to view your Bookmarks.

Hot Topics

NewsblogOT - Catch-All Pop Culture Extravaganza (April 2019)
(333 - 2:14pm, Apr 18)
Last: Hysterical & Useless

NewsblogWhite Sox-Royals benches clear, managers go after each other | The Kansas City Star
(22 - 2:11pm, Apr 18)
Last: Rusty Priske

NewsblogRed Sox injury: Where things stand with Red Sox second baseman Dustin Pedroia's knee | WEEI
(4 - 2:05pm, Apr 18)
Last: Jose is an Absurd Kahuna

Sox TherapyWe Can Be Concerned Now
(41 - 2:01pm, Apr 18)
Last: Jose is an Absurd Kahuna

NewsblogOT Soccer Thread, v.2019
(1163 - 1:57pm, Apr 18)
Last: jmurph

Gonfalon Cubs2019 Season Predictions
(76 - 1:56pm, Apr 18)
Last: Meatwad

Hall of MeritMost Meritorious Player: 1915 Discussion
(12 - 1:49pm, Apr 18)
Last: DL from MN

NewsblogOT - NBA thread (Playoffs through off-season)
(90 - 1:49pm, Apr 18)
Last: Jtsports01

NewsblogMLB umpires missed 34,294 ball-strike calls in 2018, study shows
(196 - 1:22pm, Apr 18)
Last: Never Give an Inge (Dave)

NewsblogExclusive: Giants acknowledge it’s finally time to explore moving in the fences at Oracle Park – The Athletic
(26 - 12:52pm, Apr 18)
Last: .

NewsblogPrimer Dugout (and link of the day) 4-18-2019
(10 - 12:49pm, Apr 18)
Last: CheersUnusualPlays

NewsblogWe want OMNICHATTER!, not a belly scratcher, for April 17, 2019
(65 - 12:24pm, Apr 18)
Last: The usual palaver and twaddle (Met Fan Charlie)

NewsblogBrian Cashman defends Yankees medical staff after Greg Bird injury
(14 - 12:15pm, Apr 18)
Last: The usual palaver and twaddle (Met Fan Charlie)

NewsblogContract Extension Fever Isn’t Just About Economics | FanGraphs Baseball
(51 - 12:01pm, Apr 18)
Last: .

Newsblog'Major League' turns 30: How a Dodgers hero helped turn Hollywood stars into a real baseball team
(13 - 11:21am, Apr 18)
Last: Greg Pope

Page rendered in 0.7693 seconds
46 querie(s) executed