Finally, a music reference from Repoz I don't have to look up.

For the following analysis, I'm going to rely on Win Probability Added. WPA is perhaps the most appropriate stat for investigating these questions, because it measures the impact of every play on a team's chances of winning the game. None of this "production" vs. "winning" distinction. In WPA, we measure production in terms of its impact on winning the game.

And that's where I stopped reading the article...

Finally, a music reference from Repoz I don't have to look up.

I was hoping for a West Side Story reference.

None of this "production" vs. "winning" distinction. In WPA, we measure production in terms of its impact on winning the game.

They aren't the same thing, though. Offensive production before and after August 1, by nearly any metric, would paint a much clearer picture than WPA, which adds a ton of noise to this analysis.

And that's where I stopped reading the article...

Imagine my disappointment!

Imagine my disappointment!

I was actually disappointed as I was genuinely interested in the article until that point. Next time don't use a stat that is horribly flawed.

I'm imagining it and in my imagination you are popping Zoloft like it is Pez.

If a young team succeeds it's obviously because they don't know they're not supposed to be there.

I'm imagining it and I'm surprised you are not on Zoloft.

Yes, my suffering is deep and knows no bounds.

Offensive production before and after August 1, by nearly any metric, would paint a much clearer picture than WPA, which adds a ton of noise to this analysis.

WPA/LI would mitigate much of that. I wouldn't use unweighted performance metrics because (whether people want to admit to it or not) there IS a difference between a home run that adds to a lead and one that takes you from being behind to being tied or ahead. Not *as much* of a difference as raw WPA gives you, but enough of a difference, IMO, so that you want to account for it when measuring how players react as the pressure ratchets up. WPA/LI does that.

-- MWE

I can see that, Mike. Seriously, the point of the exercise is to meet the story tellers on their own terms. I think many of them would say that yeah, production might stay the same, but what about winning? WPA measures contributions to wins. I understand why many people here don't like it, but I'm addressing a different audience with a different way of perceiving the world.

studes, that makes sense. I still think looking at it from a production point of view would shed more light on the subject. It might not be as useful in an argument with the type you're addressing, but it would give us a better idea of what's really happening, not just what the numbers say from a less-than-ideal perspective.

Either way, I was reading it more as a search for truth in numbers, and not so much as a rebuttal to a common argument. Your explanation clears up your choice of metric.

the point of the exercise is to meet the story tellers on their own terms.

...something that's absolutely necessary to gain acceptance, much of the time.

Tango's development and presentation of leverage is IMO the most important conceptual breakthrough in sabermetrics in the last 10 years - because (a) it provides a framework within which analysts can discuss what non-analysts perceive to be true, that some PAs in a game are more important than others, (b) it provides a way to do a direct comparison, and (c) it presents it in a way that is both logical and simple for non-statistically-oriented people to grasp. It targets the audience that is IMO most skeptical about the value of statistical analysis and engages that audience on their terms to a degree that other approaches have not been able to do.

-- MWE

Tango's development and presentation of leverage is IMO the most important conceptual breakthrough in sabermetrics in the last 10 years.

I don't disagree at all. You could make an argument for some fielding insights or DIPS (your other favorite subject) but LI is incredibly useful and can hopefully bridge the gap between analysts and non-analysts.

WPA measures contributions to wins. I understand why many people here don't like it, but I'm addressing a different audience with a different way of perceiving the world.

I understand that, but WPA does not measure contribution to wins.

I think LI and WPA are flawed ways to examine this problem because teams that are "too young" find all sorts of ways to lose that are hardly confined to classic late innings situations. Take a look at the MIN-NYY game from yesterday to see how this happens: young fielder makes dumb error in the middle innings, which unsettles young pitcher, who gives up runs and the team can't battle back in a tough house. In fact that whole series was a bunch of blowouts which left very few high leverage innings but clearly could be called a case of youngsters not knowing how to get it done in the big city.

For what it's worth, I think some aspects of the 'know how to win' thing might exist on one level or another, but it has more to do with the individual player than his veteranness. Some guys just love the big games more than others, but it's not something you're going to be able to quantify well.

I understand that, but WPA does not measure contribution to wins.

It does by definition.

Whatever. This seems to be a religious debate and I'll try to abstain. And don't take this personally. I got into this with Larry Mahnken once and I like him.

It does by definition.

Whatever. This seems to be a religious debate and I'll try to abstain. And don't take this personally. I got into this with Larry Mahnken once and I like him.

WPA claims to do so, but it fails. After Tango mentioned it in another thread, I read up on WPA/LI. If I understood it correctly, it does a much much better job than WPA at actually measuring contributions to winning. And I don't take anything posted on the internet personally.

And I don't take anything posted on the internet personally.

Cool. I don't think that many people do, but I'm often afraid that they do.

I think LI and WPA are flawed ways to examine this problem because teams that are "too young" find all sorts of ways to lose that are hardly confined to classic late innings situations. Take a look at the MIN-NYY game from yesterday to see how this happens: young fielder makes dumb error in the middle innings, which unsettles young pitcher, who gives up runs and the team can't battle back in a tough house. In fact that whole series was a bunch of blowouts which left very few high leverage innings but clearly could be called a case of youngsters not knowing how to get it done in the big city.

You see, this is all subjective, if you cared to pay attention and look for it, you would also see veteran players finding all sorts of "ways to lose" as well.

We've got four age groups, from youngest to oldest, with an alternating up and down pattern of performing well under pennant pressure. Is this a meaningful pattern?

This may be a futile errand, but: the savvy of a veteran has to be seen in the context of his declining athletic ability. All age groups could be about equal (indeed, as frequently noted, they pretty much are, because they're all major-league players; guys who aren't good enough yet or any more drop out of the sample). But the ratio of savvy to skillz could be much higher among older players. Hence, looking just at their "WPA Diff" or any similar clutch stat isn't going to determine the wisdom of the old saw very much, if all people mean by the old saw is that old age and deviousness will defeat youth and talent ...

I think the strange results presented could be explained by selection bias. The players who did well at the beginning of the season, particularly those in the 25-30 range, would be likely to keep getting at bats or IP after August 1st (even if they were flukey). The players who perform badly at the beginning of the season are probably more likely to get dropped (and therefore not appear). The reverse would explain the boost for the younger players (i.e. young players who would not necessarily deserve to get pulled at the beginning would get pulled quickly, but would typically appear at the end of the season in a post-August 1st call-up -- particularly after the 40-man roster expansion).

The 31-35 increase IS interesting, but is confounded because we don't know what's going on in the other age groups. I would guess that the 36-40 group would be that which is most affected by wear and tear, so I'm not shocked.

IMO, one of the critical sabermetric questions left is the censoring issue. Future playing time is often determined by past performance rather than current true talent level, so our ability to measure performance at all will be biased. In other words, certain players will be allowed to struggle at the beginning and others won't, even if they have the same true level of ability and it all depends on past performance.

I think the problem here is that WPA is a relatively young stat that hasn't really had a chance to mature and demonstrate that it "knows how to measure" at the highest levels. Give it a few more years and WPA will have the confidence and know-how to make tough measurements of contributions in clutch situations.

Primey for #23.

I think the strange results presented could be explained by selection bias.

Agreed.

...it does a much much better job than WPA at actually measuring contributions to winning.

I can understand preferring WPA/LI to WPA, and it may become preferred by the majority eventually. But it's pretty dang hard to explain just what a player's WPA/LI number represents.

This has been said before, but WPA quantifies the ups and downs of a game, and I find it fascinating for just that reason. I like looking at how players contributed, or didn't, within that context. And I think the average fan can relate to it as a result.

And that's all I'll say about the subject. :)

WPA question.

Diamondbacks vs. the Padres in San Diego. Micah Owings is pitching (and batting ninth) for the Diamondbacks. Peavy is pitching for the Padres. Huge pitcher's duel (an understatement): there is only one hit (and no walks or errors or HBPs or anything else causing runners to reach base) the entire game. Both pitchers pitch the entire game.

(1) Micah Owings hits a home run in his first at bat (in the third inning), but strikes out in his other at bats (in the sixth and ninth).

(2) Micah Owings strikes out in the third and the sixth innings, but hits a home run, breaking up the no-hitter with two outs in the ninth.

What is Micah Owings's offensive WPA for the game in situation (1)? What is it in situation (2)?

What is Micah Owings's offensive WPA for the game in situation (1)? What is it in situation (2)?

It's A LOT higher in situation 2, because the DBack's win probability went from roughly 50/50 to (I don't know off hand, 95%?)

Same fact pattern but make it in AZ instead, and in sit 2 his WPA would be even higher, since a walkoff in that situation would have made the DBack's win probability 100%.

Of course the problem is that in hindsight Owings contributed the exact same amount to each win.

Asked out of curiosity: what happens to David Ortiz's WPA when with two outs and a runner on second he singles only to see the runner get gunned down at the plate?

Meanwhile, I don't like a metric that ultimately rewards a guy for performing badly in the early innings when, with a better performance early on, his team wouldn't need his heroics late. If David Ortiz can hit home runs at will in clutch situations, let him hit them early on so that the team doesn't ever get into those clutch situations.

Asked out of curiosity: what happens to David Ortiz's WPA when with two outs and a runner on second he singles only to see the runner get gunned down at the plate?

Well, it depends on how the WPA is collected. Based on the way Fangraphs does it right now, Ortiz is penalized for the baserunner's out. There is a lot to be improved upon in the way the data is collected, including fielding breakouts.

Correct me if I'm wrong, but not only would Ortiz be penalized for the baserunner's out, but the pitcher would be also be rewarded for it rather than the fielder.

Correct me if I'm wrong, but not only would Ortiz be penalized for the baserunner's out, but the pitcher would be also be rewarded for it rather than the fielder.

Based on the way Fangraphs collects it, that's true. I don't know if it's true for Baseball Reference, or any other site that tracks it.

Back in the day, Jay Bennett tracked postseason WPA, and I believe he broke out fielding and baserunning, cause he collected the data himself based on personal observation, and not based on PBP feeds.

Based on the way Fangraphs does it right now, Ortiz is penalized for the baserunner's out.

..which is inappropriate. Ortiz should get credit for the runner going to 3rd, and the runner should get the penalty for the out at the plate. (That's the way that Bennett et. al. do it.)

but the pitcher would be also be rewarded for it rather than the fielder

Yes, but then it's also possible that a fielder should have been penalized for the hit in the first place instead of the pitcher.

I don't like a metric that ultimately rewards a guy for performing badly in the early innings when, with a better performance early on, his team wouldn't need his heroics late

You're not really the audience to whom the metric is being targeted, so you can feel free to ignore it. Again, sometimes you have to speak the language of the broader audience if you expect to be heard.

-- MWE

Of course the problem is that in hindsight Owings contributed the exact same amount to each win.

But the ninth-inning homerun was superclutch!

Seriously, I totally get why people think that WPA is a junk stat and is useless. But it's a really fascinating junk stat. Owings is going to remember his walk-off homerun more than his 6th inning one, as will fans, and WPA captures that.

WPA isn't supposed to tell you who the best player is. Disregarding a statistic because it doesn't do something that it isn't supposed to do ... that's like saying "I don't believe in OPS because it ignores baserunning."

You're not really the audience to whom the metric is being targeted, so you can feel free to ignore it. Again, sometimes you have to speak the language of the broader audience if you expect to be heard.

In my experience the "particular" audience you speak of isn't going to be anymore receptive to WPA than they are to Warp or Vorp or Baseruns or Zone rating, if it confirms what they already believe about a specific individual they'll use it the way a drunk uses a lamp post, if it contradcits what they believe, they'll go on a Murray Chass rant about how WPA or PWA or whatever is sucking all the enjoyment out of the game

Seriously, I totally get why people think that WPA is a junk stat and is useless. But it's a really fascinating junk stat.

I had the same opinion- until posters started using it as the be all and end all argument ender in MVP discussions and which player is better discussions.

In that way it's like RBI, a good, or at least interesting stat that (to me anyway) has negative value due to the way it is misused.

I had the same opinion- until posters started using it as the be all and end all argument ender in MVP discussions and which player is better discussions.

I guess I just don't see much of that. It certainly comes up in MVP discussions, where someone will make the Stathead 201 argument that "we're talking about VALUE here," but I can't remember anyone on this site using it as the lynchpin of any serious argument on ability.

In my experience the "particular" audience you speak of isn't going to be anymore receptive to WPA than they are to Warp or Vorp or Baseruns or Zone rating, if it confirms what they already believe about a specific individual they'll use it the way a drunk uses a lamp post, if it contradcits what they believe, they'll go on a Murray Chass rant about how WPA or PWA or whatever is sucking all the enjoyment out of the game

If you present leverage rather than WPA, people get it. They get that certain plate appearances are twice as valuable as a typical plate appearance, while others are half as valuable.

-- MWE

In my experience the "particular" audience you speak of isn't going to be anymore receptive to WPA than they are to Warp or Vorp or Baseruns or Zone rating, if it confirms what they already believe about a specific individual they'll use it the way a drunk uses a lamp post, if it contradcits what they believe, they'll go on a Murray Chass rant about how WPA or PWA or whatever is sucking all the enjoyment out of the game.

That's mostly been my experience, too. But we all have our windmills. In particular, THT is oriented toward the sort fan who is open to learning more about the game.

If it's supposed to speak to the broader base of baseball fans (i.e. dumber ones), it shouldn't be named Win Probability Added. I don't think many of the droolers out there can get their heads around a stat that employs a five syllable word. If it's really supposed to bridge the gap then it ought to just be called Clutch Rating or something.

Mike/13: thanks for the kind words.

For those who haven't checked, Fangraphs shows LI numbers for each play, game, and seasonal totals for all players going back a few decades (so far). It's really cool to see how the leverage numbers of pitchers change. As an example, and focus on the column "pLI" near the bottom of the page: Keith Foulke

That 2002 season, where he was pulled from the bullpen (and you see it from his LI plumetting so much that season), and his clutch score being so negative. But, his WPA/LI, which gives a better view as to how he was actually pitching, really shows him to be the same year after year.

That's what WPA and LI deliver for you, something that the traditional lines don't: the LI shows that he wasn't used in as high leverage. His WPA shows that he had much less impact than in his surrounding years, and yet his WPA/LI shows that he was really the same pitcher, and his clutch score shows that he pitched bad in some crucial situations, which probably led to his demotion.

***

Chris/16: "I think LI and WPA are flawed ways "

LI has nothing to do with your concern. Your WPA concern is legitimate, but your LI one is not.

***

Bob/21: agreed.

***

studes/25: agreed, it is harder to explain WPA/LI compared to WPA.

***

The issue of baserunning and fielding in WPA is one of implementation, not the framework. The more granular you have the data, and the more programming you want to do, the more you are going to represent it to what it should. Obviously, the hitter should not get penalized for the runner being thrown out. That's not a WPA problem, but either a data collection or programming problem.

***

As for the HR in the 1st or 9th: the assumption of win expectancy, WE, (as in life) is that everything in the future is unknown, and therefore, your expectancy is that average things will happen. So, hitting a HR in the 3rd increases your win expectancy by, I dunno, +.130 (say from .500 to .630), because you have a certain expectancy of what will happen in the following 6 innings. So, the WE represents how much you are willing to bet on the game's outcome at that point in time, and the change in WE (i.e., WPA) simply pays off on that bet.

In the 9th inning, you are "all in". You may have say won most of the poker hands before that, but if someone goes all in, and he wins, you lose.

That's all WPA is... an accounting of money. In real-life terms, as studes put it, it simply tracks your emotions, and quantifies it.

So, I can see why WPA gets a bad rap... either because it is misunderstood or misapplied. OBP doesn't get a bad rap, even though it weights a walk and HR as exactly 1. Its limitations are self-evident, and therefore, no explanation needed. With WPA, we have to work harder at explaining what it is and what it isn't.

Belittling WPA, as a stat, makes no sense, because it represents what it purports to represent. Nothing more.

I support Tango on this one. I think WPA, properly understood and properly used, is a useful statistic. The title says exactly what it is. This is as opposed to the statistic I dislike the most, pitcher abuse points, in large part because the title makes an inappropriate value judgment. It is for this reason that it should definitely not be called something like clutch rating - clutch is a value judgment, and WPA isn't really a rating anyway.

WPA is relatively easy to understand, and measures something tangible. As do RBI. The fact that a statistic is misused does not make it a junk statistic.

As for the HR in the 1st or 9th: the assumption of win expectancy, WE, (as in life) is that everything in the future is unknown, and therefore, your expectancy is that average things will happen. So, hitting a HR in the 3rd increases your win expectancy by, I dunno, +.130 (say from .500 to .630), because you have a certain expectancy of what will happen in the following 6 innings. So, the WE represents how much you are willing to bet on the game's outcome at that point in time, and the change in WE (i.e., WPA) simply pays off on that bet.

In the 9th inning, you are "all in". You may have say won most of the poker hands before that, but if someone goes all in, and he wins, you lose.

That's all WPA is... an accounting of money. In real-life terms, as studes put it, it simply tracks your emotions, and quantifies it.

I understand this and agree with this.

Belittling WPA, as a stat, makes no sense, because it represents what it purports to represent. Nothing more.

I do not agree with this. Several people have stated that WPA measures contributions to wins. It does not do that. As I understand it, WPA/LI does that. As long as people continue to claim that WPA measures contribution to wins, I will keep calling it a broken stat.

I think you can say that WPA does measure contributions to wins, within the context of how it's accumulated (real time context). You may disagree with the validity of the context, but there is more than one way to measure something like "contribution to wins."

Right, what studes/43 says. It's exactly what the betting line shows. If it's a 0-0 game in the bottom of the 9th inning with 2 outs, then both sides have anted up say 500$. If you get a HR, guess what... you end up with 1000$ in your pocket, the loser gets nothing. We "credit" that last winning poker hand with making you a 500$ profit. Does it make sense that an Ace pair gets to count for +500$, while a straight only made you 25$ early in the game, when you were still at dollar-ante? Well, at the time the bet was made, that's what happened. That straight only contributed 25$ toward your winnings. That ace-pair ended up paying off 500$.

This is all it is: we know the amount of payoff for that event, and we know what event took place. We match the two.

Now, you can argue that most of that was pure random variation, and that an ace-pair will normally just make you a 20$ profit (which is what WPA/LI would give you). Fine. But, you actually ended up with a 500$ profit. If you don't wish to assign it to the ace-pair, you have to assign it somewhere. You can assign it to some "timing" bucket. Whatever you wish to do, is fine.

But, you did have a 500$ profit recorded.

Therefore, what we can agree with is that 500$ was earned, but what your valid point would be that you don't want to assign that 500$ to a particular event, because that event really was just "right place right time". No problem. Create a "timing" bucket, and we can move on.

You know, Tango, that's an interesting idea. For each game, set aside WPA-WPA/LI in a bucket, then divvy it up among players after the game by some metric (perhaps WPA/LI itself). You could call that "Neutral WPA" or something like that (a contradiction in terms!). The value of that metric would be that it would add up to each team's total wins and losses but it would also address Randy's issues.

In some respect, I'd prefer to leave the timing bucket alone, because it is what it is. At the end of the season, you'd see how much of the whole thing was dependent on the timing.

On the other hand, WPA does serve a purpose, if you look at it from the pitcher. A pitcher is the one who puts himself in the position he finds himself. So, any bad timing, he'd have to answer to.

As for relievers, Mo sort of puts himself in LI=2 situations because he is Mo the Great.

You could in one sense carry two timing buckets, one for pitchers and one for nonpitchers. For the nonpitchers, they share the timing bucket, and for pitchers, they own theirs.

You can't have a reliever with a 1.00 ERA who only pitches in mopup situations "earn" alot of money. He's basically Steve Jobs in 1978... he's not yet STEVE JOBS, even if he worked as hard in 1978 as he did in 1984. The money just wasn't there. Or someone who keeps winning penny-ante pots, but is not put in the high-stakes games.

Does WPA separate the contribution of a pitcher vs. the contribution of the defense on any play that results in a loss? Basically, how does WPA account for unearned runs? My impression just by looking at some game scores on fangraphs is that the pitcher gets saddled with the negative WPA even if the defense fumbled the ball around and led to unearned runs... and the loss. Is that true? Or does WPA separate the pitching component from the defensive component?

Again, please separate the framework of WPA from the implementation at Fangraphs.

As for your particular question, I have answered it on my wiki, at Question 40.