Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Baseball Newsstand > Discussion
Baseball Primer Newsblog
— The Best News Links from the Baseball Newsstand

Tuesday, August 23, 2011

Hughson: The Use Of Sabermetric Stats Makes Me Uncomfortable

“accuracy - not flakiness” Handles for sale! Get your red hot handles!

Skill-Interactive Earned Run Average (SIERA)

Skill-Interactive Earned Run Average estimates ERA through walk rate, strikeout rate and ground ball rate, eliminating the effects of park, defense and luck.  How is it calculated you ask?

SIERA = 6.145 – 16.986*(SO/PA) + 11.434*(BB/PA) – 1.858*((GB-FB-PU)/PA) + 7.653*((SO/PA)^2) +/– 6.664*(((GB-FB-PU)/PA)^2) + 10.130*(SO/PA)*((GB-FB-PU)/PA) – 5.195*(BB/PA)*((GB-FB-PU)/PA)

The statistic is so convoluted that Baseball Prospectus removed SIERA from their toolbox of stats.  That’s good enough for me.

...Ideally, what I want from my statistics are black and white outcomes.  That’s what I love about OBP – either the player got on base or he didn’t.  There’s no sample size error, no assumption of universal causation, no convoluted formula.  Putting statistics into a greater context allows us to fully flesh out what a player brings to the table. I recognize that the gripes I present about the stats listed above may be perceived by some as minor.  As I said before, I realize that the stats I am uncomfortable with still have value, however, I want accuracy from my stats - not flakiness.  It is when these stats are perceived as the be-all and end all that makes me uncomfortable.

Repoz Posted: August 23, 2011 at 12:32 PM | 29 comment(s) Login to Bookmark
  Tags: community, fantasy baseball, history, projections, sabermetrics, site news, special topics

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. cHiEf iMpaCt oFfiCEr JE Posted: August 23, 2011 at 12:50 PM (#3906656)
Is my thinking flawed? Do I have a misunderstanding of one or more sabermetric stats? Leave a note in the comment section to try to convince me otherwise.

Comments are closed.

Oops.
   2. BDC Posted: August 23, 2011 at 01:29 PM (#3906671)
This is what I always loved about the simplicity of Pancake Flops. (3*2B + HR^1.3 - outs)/GIDP, you can practically figure that in your head. Josh Hamilton has -20.46 this year.
   3. Benji Gil Gamesh VII - The Opt-Out Awakens Posted: August 23, 2011 at 01:51 PM (#3906681)
Callum, show us on the Bill James doll where OPS makes you uncomfortable.
   4. RoyalsRetro (AG#1F) Posted: August 23, 2011 at 01:54 PM (#3906685)
All of the statistics listed above make me uncomfortable when used on their own.


Agree. The examples he provides are pretty good critiques of those stats I think. They need to be viewed in context.

When analyzing a player I like to couple conventional statistics with sabermetric ones


Um..okay, I guess it depends.

. Range factor is better when paired with a statistic like fielding percentage.


Um...no.

Seems like his standards for what stats can give him is asking too much - one stat that on its own can give you complete and accurate value in any context.
   5. salvomania Posted: August 23, 2011 at 02:09 PM (#3906698)
Also, aren't the examples given (OBP and sIERA) useful for different types of analysis---what happened vs. what we think is more likely to happen going forward?
   6. SG Posted: August 23, 2011 at 02:17 PM (#3906708)
I disagree with some of the things in this post, but overall it's not a bad one. He clearly details why he doesn't like certain stats and it's pretty obvious he's done more than cursory research to try and understand them. He also stays away from the silly jabs that define the typical "stats bad" articles.

One thing I think he's not quite understanding is WAR when he says;

Lastly, the problem I have with WAR is semantics – I’m not comfortable with the fact that ”wins” can be assigned to an individual player in a team game such as baseball.


Technically speaking, WAR is more like a bunch of pieces of wins credited to players. A player isn't getting credit for any specific win, he's getting credit for how his total contributions have combined to his team winning or losing over the course of the season. Kind of like win shares using a different scale and different value algorithms.
   7. zack Posted: August 23, 2011 at 02:45 PM (#3906738)
A lot of people have problems with stats that are estimators, not descriptive (population) statistics, like UZR vs. fielding percentage, and that's understandable. But the more involved answer is that even the descriptive statistics are just a sample of what we usually care about, which is the player's actual ability. I don't really care whether Billy Butler got on base 150 times or 200 times, I want to know how good he is at baseball, and that information helps me answer that question.
   8. whoisalhedges Posted: August 23, 2011 at 03:24 PM (#3906782)
Technically speaking, WAR is more like a bunch of pieces of wins credited to players. A player isn't getting credit for any specific win, he's getting credit for how his total contributions have combined to his team winning or losing over the course of the season. Kind of like win shares using a different scale and different value algorithms.

Of course, and despite some flaws with both, this is what WS and WAR both get right. Sure, you can look at the complete game log; and if you've seen the game you can make some sort of judgment as to how each player fielded his position (and went out of position to make plays). You can look at dozens of different statistics to make your in-depth analysis of a player's performance, but this is what it all comes down to: the point in baseball is to win the game.

Pre-WS, pre-WAR, wins were granted to pitchers and pitchers alone. Well, if a pitcher allows 5-6 runs a game, but his team scores 8-9, he'll probably have a decent number of wins -- but we all know that doesn't make him a good pitcher. Why should he get credit for the W when it's really his batters who won the game.

No omnibus metric will ever be perfect, but I think WS and WAR are a step in the right direction (or, given that WS ignores replacement value, two right steps in two different directions). The point is to win. And just because one of your relievers only gives up three runs in an inning while the others were busy giving up four (yet your hitters scored thirty) doesn't mean that pitcher who had a shitty day, just slightly less bad than anyone else, should get the credit.

All ballplayers contribute to wins. Ultimately, that's how these tools work, to try and define that contribution. WS goes so far as to directly make it team-based, 3 WS to each win. And although I am NOT a mathemetician of any stripe, I actually think this works for a good number of teams out there. Intuitively, I don't think it's terribly relevant to major outliers: the '62 Mets, the '06 Cubs, or teams whose records differ wildly from their Pythag. But for many teams, yeah, WS can probably give a pretty good sense of a player's actual contributions.

WAR, of course, brings in replacement level -- I think WS may actually be better for season-end awards (what actually happened), while WAR is certainly better for "should this guy be on the team at all?" If a GM/managerial staff had a really good version of WAR, it could do wonders to help them construct their team. Hey, there's two free agents out there, a 3B and a 2B, and we have holes as both of those positions. They're both asking for the same money -- which one should I sign, and which one should I promote from the minors? That kinda thing, I think WS is pretty well useless for that.
   9. PreservedFish Posted: August 23, 2011 at 03:36 PM (#3906795)
As I said before, I realize that the stats I am uncomfortable with still have value, however, I want accuracy from my stats - not flakiness. It is when these stats are perceived as the be-all and end all that makes me uncomfortable.


I'm with him.
   10. Fancy Pants Handle struck out swinging Posted: August 23, 2011 at 03:40 PM (#3906802)
Callum, show us on the Bill James doll where OPS makes you uncomfortable.

Damn you. The only reason I clicked on this thread was to make thatjoke...
   11. TDF, trained monkey Posted: August 23, 2011 at 04:26 PM (#3906845)
Count me among those who generally agree with the author.

I also share his specific complaint about defensive metrics, but I think he explains it poorly. We're told it takes 3 year's worth of data to make any conclusions; in his career, Jay Bruce has been -11, +3, +17, and -11 runs (per BBRef). Granted, he had fewer than 1000 innings in each of the first 2 seasons and there's still 5 weeks left in this one, but how good of a fielder is Bruce? Is he as excellent as his reputation? Is he about average (-2 runs for his career)? Is '10 an outlier and he's really bad?

I'd argue you can't assume a player has the same talent level over 3 years - younger players (like Bruce) should learn their positions better while older players probably can't get to as many balls as before. How am I supposed to use defensive metrics?
   12. callum17 Posted: August 23, 2011 at 04:32 PM (#3906850)
Please accept my apologies. I've been having some issues with Wordpress and it has been automatically been disabling comments. That has since been remedied! Flame away.
   13. Random Transaction Generator Posted: August 23, 2011 at 04:40 PM (#3906860)
Lastly, the problem I have with WAR is semantics – I’m not comfortable with the fact that ”wins” can be assigned to an individual player in a team game such as baseball.


They've been assigning wins to pitchers for ever. I assume he'll have no problem ignoring that stat going forward.
(As he should...)
   14. callum17 Posted: August 23, 2011 at 04:58 PM (#3906878)
I have no problem whatsoever ignoring Wins for pitchers :)
   15. cardsfanboy Posted: August 23, 2011 at 05:28 PM (#3906912)
As I said before, I realize that the stats I am uncomfortable with still have value, however, I want accuracy from my stats - not flakiness. It is when these stats are perceived as the be-all and end all that makes me uncomfortable.


I'm with PF and the writer, but even the stats that I'm truly attached to, have flaky components. ERA has the whole decide what happens after the error for determining earned runs that is extremely flaky. It should be an impossibility for a homerun to be an unearned run, the men on base I could understand, but the actual homerun should always be an earned run. How about not assuming the double play for errors? Or the really weird nonsense of making a sacrifice fly not an at bat. You can't claim that those aren't flaky. And of course as mentioned the wins stat.
   16. Dan The Mediocre is one of "the rest" Posted: August 23, 2011 at 05:34 PM (#3906917)
Lastly, the problem I have with WAR is semantics – I’m not comfortable with the fact that ”wins” can be assigned to an individual player in a team game such as baseball.


This is pretty wrong. No individual win is assigned to anyone - it's supposed to be an average of how many wins a team with Player X had over how many it would have had with a player that can be randomly found in AAA. Certainly the replacement level can be argued (and always is), but runs/wins compared to a baseline is the only way to figure out a tangible way a player made his team better.
   17. zenbitz Posted: August 23, 2011 at 05:42 PM (#3906922)
Over his 4 year career, Jay Bruce has an average RC (batting) of ~68 runs.

He has been -12, -19, +20, +12 (per BBRef).

How good a hitter is Bruce?
   18. RoyalsRetro (AG#1F) Posted: August 23, 2011 at 06:25 PM (#3906966)

This is pretty wrong. No individual win is assigned to anyone - it's supposed to be an average of how many wins a team with Player X had over how many it would have had with a player that can be randomly found in AAA. Certainly the replacement level can be argued (and always is), but runs/wins compared to a baseline is the only way to figure out a tangible way a player made his team better.


Right, but I think "Wins" can be confusing in "Wins Above Replacement" when it could be "Widgets Above Replacement" in the sense that we only want a value of what that player is compared to a replacement level player. I think too many unfamiliar with the stat means it literally translates to win, so when they see a guy go 1-4, with a productive out, and the one hit driving in the game winning run late in the game, he will argue "That is 1 WAR right there!". And yes, I've had that argued to me.
   19. Shockliacci Posted: August 23, 2011 at 06:47 PM (#3906990)
Can I just say that I hate the phrase "be-all and end-all?" NOTHING is the "be all and end all" of ANYTHING. It is completely useless to point it out and is an implied straw man argument.

Worst thing Shakespeare ever did.
   20. Weekly Journalist_ Posted: August 23, 2011 at 06:54 PM (#3906998)
Royce: Your point just further reinforces Shakespeares genius. Of course nothing its the be all end all, which is why.Duncan's murder isnt the end of the play
   21. Darren Posted: August 23, 2011 at 07:21 PM (#3907022)
That’s what I love about OBP – either the player got on base or he didn’t.


Unless there was an error. Or a fielder's choice. Or a sacrifice. Or a missed third strike.
   22. PreservedFish Posted: August 23, 2011 at 07:25 PM (#3907024)
NOTHING is the "be all and end all" of ANYTHING. It is completely useless to point it out and is an implied straw man argument.


It's just a phrase. Are you mad at him because "all's well that ends well" isn't true?
   23. Bourbon Samurai stays in the fight Posted: August 23, 2011 at 08:19 PM (#3907107)
Does it seem to anyone else that "Mr. President" is missing from the end of the title?
   24. Bourbon Samurai stays in the fight Posted: August 23, 2011 at 08:20 PM (#3907109)
Although now I actually read the article and its pretty good.
   25. TDF, trained monkey Posted: August 23, 2011 at 08:36 PM (#3907126)
Over his 4 year career, Jay Bruce has an average RC (batting) of ~68 runs.

He has been -12, -19, +20, +12 (per BBRef).

How good a hitter is Bruce?
We say, with confidence, that Bruce has become a pretty good hitter; we have that confidence because over a statistically significant amount of time (almost 2 full seasons) he's been consistantly good (although, frankly, I don't see any RC numbers at BBRef).
   26. cardsfanboy Posted: August 23, 2011 at 09:04 PM (#3907162)
(although, frankly, I don't see any RC numbers at BBRef).


click on more stats, They don't have it in pi, but in each players page they have both RC and RC/27.
   27. Ron J Posted: August 23, 2011 at 09:34 PM (#3907197)
#25 And yet the best we can do in terms of estimating how well Bruce will hit next year (assuming he's healthy) with all of the information at hand is 10 runs. And that's probably being generous. It's not Bruce's age or anything. Best I can tell well established players aren't a heck of a lot more predictable.
   28. zenbitz Posted: August 23, 2011 at 09:37 PM (#3907199)
@25 - the point is that offense - especially in a non-rate context - varies just as much as UZR, if not more so.

BA (and with it OBP and SLG) has a pretty high variance.
   29. Ron J Posted: August 23, 2011 at 09:51 PM (#3907207)
#28 I'm pretty sure you've seen this before, but:

The standard deviation of BA of players with 300+ plate appearances in consecutive seasons is just over 30 points of BA. (And around 35 for OBP and roughly twice that for SLG).

I've also looked at consecutive months: using only players with 90+ PAs in both months (which should get most of the injured players out) 49 points of BA, 57 point of OBP and 117 points of SLG

You must be Registered and Logged In to post comments.

 

 

<< Back to main

BBTF Partner

Dynasty League Baseball

Support BBTF

donate

Thanks to
A triple short of the cycle
for his generous support.

You must be logged in to view your Bookmarks.

Page rendered in 0.2748 seconds
45 querie(s) executed