|
|
|
|
Baseball Primer Newsblog— The Best News Links from the Baseball Newsstand
Friday, November 13, 2009
The Monsters Are Due on Tuckman’s Stage!
5. What are some of the biggest misconceptions in the internet stat community? If you could do away with one trendy cliche or assumption, what would it be?
Well, I think the community’s main problem is hubris combined with a groupthink attitude. Ken Rosenthal discussed this in a recent column and the sabermetric community chose to chastise rather than heed his point. This is compounded by the fact that those conducting and consuming the analysis aren’t adequately familiar with the employed analytical methods and so bad analysis sometimes becomes part of the group belief. For example, the other day I pointed out an excellent study by economists Jahn Hakes and Skip Sauer that examined baseball’s labor market at Baseball Think Factory . A commenter responded “Tangotiger hates the Hakes & Sauer paper”; I guess I was supposed to defer to him. Anyway, I followed the link and the analysis conducted by the pseudonymous sabermetric icon doesn’t refute the findings at all. He’s plugging in extreme values into a model to make absurd predictions outside the sample for a model that the authors are acknowledging is out-of-whack with what should be. For some reason, this damns the model. I have seen Jahn and Skip present their work several times in front of many economists who are well-versed in the techniques used. It’s been vetted by skilled referees and editors and published in respected academic journals. I’ve read their work closely and talked to them about it. Yet, what bothers me is not that someone reaches an erroneous conclusion, but that the commenters wholehearted embrace the flawed critique, which it is later parroted across the Internet. No attempt is made to contact the authors, or submit a response to the journal that published the article—a common practice when flaws are discovered after publication. That’s not what this is about, it’s some sort of status game—chest thumping at a safe distance. Sabermetrics (with a big S) has become a club focused on rhetoric, not a serious research program.
|
Bookmarks
You must be logged in to view your Bookmarks.
Hot Topics
Newsblog: Boston.com: Curt Schilling’s 38 Studios lays off all staff (44 - 4:58am, May 25)Last: Obi One Kenobi NilNewsblog: Shawn Green to play for Israel in World Baseball Classic (10 - 4:57am, May 25)Last: SnowboyNewsblog: Wins Above Replacement: Distribution and Rarity of Talent 2011 - Beyond the Box Score (9 - 4:18am, May 25)Last: bobmNewsblog: Greenberg: Cubs' Ricketts decries proposal (749 - 3:19am, May 25)Last: Greg (U)KNewsblog: Dodgers want to host NHL's Winter Classic (15 - 3:07am, May 25)Last: Greg (U)KNewsblog: OT: NBA Monthly Thread, May 2012 (1771 - 3:02am, May 25)Last: robinredNewsblog: Neyer: New Yankee Stadium: A Review (74 - 2:00am, May 25)Last: Dag Nabbit apealing [sic] his own check swingNewsblog: OT: NHL Playoff Thread (1731 - 1:45am, May 25)Last: baudibNewsblog: Ross Newhan: Freeing a Son From His Father's Words (5 - 1:44am, May 25)Last: Curse of the AndinoNewsblog: 12 Baseball Feats That Only Happened Once (24 - 1:43am, May 25)Last: Drexl SpiveyNewsblog: Major League Baseball named Sports League of the Year at Sports Business Awards (10 - 12:40am, May 25)Last: LunkusNewsblog: Bud Selig -- No need for more MLB replay for now - ESPN (64 - 12:38am, May 25)Last: Sunday silenceNewsblog: Cardinals unveil latest Ballpark Village plan (4 - 12:36am, May 25)Last: Ivan Grushenko of Hong KongNewsblog: Roy Halladay bobblehead with glove on wrong hand selling on MLB.com (8 - 12:10am, May 25)Last: The District AttorneyNewsblog: Kelley: Time for Mariners to waive Chone Figgins, play the kids (35 - 11:41pm, May 24)Last: Johnny Slick
|
|
Reader Comments and Retorts
Go to end of page
Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.
2. This isn't an academic symposium. It's a baseball geek message board. WTF do these guys want me to do? Spend a month on a post and have it peer reviewed? That's going to make for a very lonely internet. Which, of course, may be a good thing, though it's not going to help me get through my work day.
I also very much disagree with JC's statement that Sabermetrics (with a big S) has become a club focused on rhetoric, not a serious research program. It is notable that the vast majority of "Sabermetric innovations" - such as DIPS theory, statistical evaluation of defense, linear weights, etc - were not first published in peer reviewed journals but rather were invented in someone's garage or basement. To this day, that trend of innovation continues (such as with the analysis of PitchFX pitch-level data, to give an example). Of course, JC would not consider much of this as a "serious research program" because it does not stand up to the statistical and theoretical rigor it would need to for inclusion in a peer reviewed journal.
I'm not sure why we're supposed to defer to economists when looking at baseball transactions. When did they become authorities on the subject?
boochit--we do NOT have a groupthink attitude
(and everyone here would agree with me)
Go stab a neck!
One finds a sharp reduction of snark on The Book Blog, for example.
In looking at the original thread, the poster did include a substantive comment so it wasn't merely an appeal to authority.
Anyway, I followed the link and the analysis conducted by the pseudonymous sabermetric icon doesn’t refute the findings at all. He’s plugging in extreme values into a model to make absurd predictions outside the sample for a model that the authors are acknowledging is out-of-whack with what should be. For some reason, this damns the model.
Has tangotiger formulated a response?
Yet, what bothers me is not that someone reaches an erroneous conclusion, but that the commenters wholehearted embrace the flawed critique, which it is later parroted across the Internet.
There were 3 comments after "Tangotiger hates the Hakes & Sauer paper" post, none of them were about the Hakes and Sauer paper. I can't tell if it was parroted across the Internet though.
EDIT: Ah cain too spale!
Well, Bradbury could have tried to respond to the criticisms of the paper made in the original thread, if he supports it so strongly, since many of them seem to be valid and were not presented in a snarky manner. Instead he basically decided to resort to name calling. Wtf do you expect the responses in this thread to be?
Look! I just snarked on Steve Treder!
Prof A - Here is my model. See it works!
Post-Doc B - Your model sucks! Mine is better!
Prof C - Prof A's model does a good job because it defines its parameters carefully, and demonstrably works within them.
Non-academic media commentator/graduate student - Prof C is a wanker! Post-Doc B ROOLS!
Prof C - Why should I defer to Post-Doc B? He hasn't got tenure. Those who support him are rude and ignorant. I'll be in my office.
True. And by and large, the authors of those methods welcome constructive criticism and "peer review" of their methods.
The broader issue isn't with the people who are actually doing sabermetric research, but with the larger community that buys into the research uncritically, and elevates the researchers to something akin to oracles. Tango's not always right (and he'd be the first one to tell you that), and just because he says something critical about Hakes and Sauer doesn't mean that his criticism is valid.
-- MWE
True; but I think this is something of a strawman. Tango and mgl and the like get plenty of criticism around here, and there are very few appeals to authority.
If anyone's appealing to authority, it's Bradbury, with his talk of "peer-review" and "published in respected journals".
Certainly. But how different is this dynamic from that of, well, just about any other community? "Authorities" in academia, business economics, stock-picking, you name it, are too often followed uncritically.
Penis penis penis!
Vagina?
Penis penis penis!
Is that good or bad?
Here's a link to the BTF comment thread which offended JC
Here's a link to Tangotiger's post on his blog arguing against Hakes & Sauer, many good points are made not only in the post, but in the comments as well.
First a disclaimer, I'm neither a statistician nor an economist, so my understanding of the issues may be faulty.
I have two objections to Hakes & Sauer. One, they, like all other academics writing about baseball, ignore all research done by amateur sabermetricians. Two, they throw both rate stats and counting stats like PA, and correlated stats like OBP and SLG into the same regression. Maybe JC is happy that H&S;try to reinvent the wheel with sloppy work like that and would call it "avoiding groupthink", but I think it just makes H&S;'s conclusions worthless.
JC claimed that my objections to H&S;were simple appeals to authority of Tangotiger. In this case, I (as far as I can tell) actually understood Tangotiger's objections and agreed with them. If I did though had to appeal to authority, JC Bradbury would not one I would trust to be correct if I didn't understand his reasoning.
I've been trying to figure out why JC was offended by a blogpost comment. After all, he's been writing on the Web for a long time, and so can't be a thin-skinned person. I think it is because he is an economist and thus is infected with the arrogance of the economics profession. In recent years, economists started believing that they could take their statistical toolbox and apply it to analyze any kind of data without first learning about the subject domain. They believe that they can thus reach valid conclusions, even overriding the opinion of domain experts, but since they didn't bother to learn about the subject first, they often just make fools of themselves. But, god forbid, someone point that out, they are just perpetuating hidebound groupthink. Regression says so!
Here are some examples of economists blundering about with sloppy statistical analysis on non-economic topics:
Kovash/Levitt wrote a paper saying that pitchers throw too many fastballs, but basically consider balls or called or swinging strikes to be of no account.
In Freakonomics, Levitt claims that drunk driving is better than drunk walking. Here's why he's wrong
And speaking of Freakonomics, let's not forget their misleading and sloppy global warming chapter.
Sorry JC, being an economist doesn't mean that you get to ignore research of other people.
That looks like the output of some thoughtless number-crunching, right down to the multiplicative constant at the end. I hope the paper made more effort to control for the correlation between SLG and OBP than this formula suggests.
I don't know if the larger community buys into the research uncritically and elevates certain researchers accordingly, but to a certain degree, we all rely on authority figures for knowledge simply because not all of us have the necessary knowledge base to evaluate the research ourselves whether it's in the medical, economic, and sabermetrics field. So if an established medical authority says that treatment X will have a X% of fixing ailment X, then I'm willing to defer to their opinion or at least seek a second opinion. So I'm not going to automatically think Tango's criticism is automatically valid but I'm more willing to trust his opinion given an established track record.
I dunno. Show of hands?
Vagina?
Cites, please.
Jeez, I read "cites" as something else, initially.
So basically, they are physicists who are bad at math?
CLINTS?
That's true, but his comments and the comments here bring up an issue that is noteworthy. I'm not a PhD in statistics but I know my way around it a bit. I see a lot of posters here tossing around statistical concepts in a way that makes me think that they don't remember their Stats 101 course. For example, "small sample size" is frequently presented as some sort of QED that an observation with a small n is invalid. Data with a small n may still yield results that are statistically significant, it's just harder to demonstrate when hypothesis testing.
This happens all over the place and I do it too. But it is troubling when arguments are summarily dismissed without examination and often with a certain smug superiority by people who seem to misunderstand what they're talking about.
Try suggesting an AL MVP candidate other than Mauer sometime.
Yes, physics PH.D's did so well on Wall Street.
As a professional economist, I feel like I should be offended by this. But I just can't be. It's accurate.
But that's my favorite part of my job! (I kid, I kid)
Seriously, I'm a big fan of the Levitt/Becker school that views economics as being a way of studying anything under the sun (my Master's degree is from the University of Chicago). But I do think that this is a perfectly valid criticism of us - economics is a wonderful TOOL for evaluating lots of things - baseball, crime, etc. - but it's very, very important for economists to defer to the experts in those areas (and, as weird as it may sound to an academician, Tangotiger is pretty clearly an expert in baseball analysis) so that they're sure that they're using that tool correctly.
Sure, but what I'm getting at is this: in what way is this behavior particular to the sabermetric community? Is what's being described here anything other than one of the less attractive elements of human nature?
A bunch of people pointed out that Greinke was more valuable than Mauer. There were some articles in support of Zobrist as well
I think GGC is referring to the thread here that arose from one of these.
I think GGC missed the joke in the post he quoted. Also, I believe Crashburn, who wrote the article supporting Zobrist, changed his stance after some discussion in that thread. Seems to be a poor example of groupthink.
Or apparently it's offensive to him for someone to present a dissenting viewpoint.
I guess it's more of an issue in the sabermetric community because:
1. It's a community that values the use of data and analytic tools to discern categorical differences and distinctions.
2. It's a community that communicates most often via the Internet, where nuance is often lost and is certainly often unappreciated. Indeed, it's a medium where volume and bluster are often mistaken for authority.
I started making other points, but it came down to my mother's comments to me about jumping off of bridges if everyone else was doing it. Why?
Non-saber analysis is hooted at if it is patently full of ####; so should saber analysis. But it'd be nice if the hooters were clad in something other than the emperor's new clothes.
Too many clothing metaphors, Smitty'd be apoplectic.
...or that the sky isn't blue, too. ;-)
He's become a parody. That post above describes him exactly. "This is compounded by the fact that those conducting and consuming the analysis aren’t adequately familiar with the employed analytical methods and so bad analysis sometimes becomes part of the group belief." This describes him exactly.
He's not a human being he's a broken robot that just keeps repeating the same tired old formulas, "econometrics is life, econometrics, is life, econometrics is life." It's time for him to be decommissioned and rehabilitated as something useful.
Until then, I'm just a monkey on a web site.
LOL
This is a frightening quote. I looked at Tango's thread that ekogan linked to, and man, is that a terrible regression equation. Throwing a hodgepodge of rate and counting stuff together, and no consideration of co-variance. Sure, a statistical package will give you coefficients for all that. But it seems such a lazy way to build a model. I would hope that people could do better than that. If this is par for the course for analysis in academic journals, then there is quite a bit of room for improvement in the analysis of every field.
Yeah, once in a while it's good to let Gaelan off the leash. I call it, Bringing Out the Gimp.
(I kid, G. I kid!)
Bill James has said his work is heavily influenced by the Chicago school of economics. I see a lot of similarity in the general approach between Levitt and James.
That wasn't a rant. It's what I believe. Bradbury has never, in the history of BTF, written something worth reading. If he were merely incompetent it would be a step up. But he isn't in the universe of incompetence since he cannot even understand the terms by which competence is measured.
I could do better than that, in Excel. And I haven't done any serious regression work in 15 years.
Yep, this goes back to exactly what I was saying up in #39. I do econometrics for a living; it's a wonderful tool. But GIGO. What's wonderful about econometrics and economics is that it provides a way to quantify things, but an expert in econometric techniques isn't the person that you need to run that model by. You need to run that model by an expert in baseball analysis, to make sure you're including the right variables and measuring them correctly.
Just off the top of my head, the two obvious problems that I see with the formula in #29 are that pre-arb and free-agent players should be modeled with completely separate equations - they're not in the same market - and you need to remove batting average from either SLG or OBP (or I would probably remove it from both and include it as a separate variable). But unless you happen to talk to an econometrician who's also a sabermetrician / serious baseball fan, there's no reason to expect an economist to come up with either of those suggestions.
Bill James has the soul of a philosopher. It permeates all his best work and has nothing to do with the methods he uses. He is the opposite of Bradbury who is all method no thought.
Sounds delicious.
EDIT: Or maybe he's JCBradbury, actually. You'll know him when you see him. Whatever the case, he lives in metro Atlanta, so he wouldn't be in DC.
That's almost enough for me to unblock him. Not quite, though. I assume he's just frothing at the keyboard & making sweeping pronouncements in the name of philosophers who've been dead & irrelevant for 300 years, as per usual. *yawn*
No, they're different people, although I think Bradbury does post here on occasion (maybe not after this thread).
he did indeed
I can't think of two people more different than Bradbury and JC in DC.
I put this out there to say I'm sympathetic to both sides.
I have not read the paper mentioned in a while, but did read Tango's analysis. I agree with Tango that OBP and SLG are very correlated, and multicollinearity could cause a problem.
I'm not sure about the critique of the extreme values. First, regressions are not designed to extropolate for extreme values that do not occur with regularity. If there is only one outfielder with .250/.250 OBP/SLG then the model is not going to predict that as well. The goal of regression to provide a model that on average that the prediction has the lowest variance.
For fun let's see we can find one of these players. I don't have b-r PI, so someone if they want can look up players similar to Tango's extremes. I did try to find one outfielder that met Tango's descriptions. I looked up this year's hacking mass results, found that Willy Taveras hit .275/.285 with 437 PA and made 2.25 million. The model predicted Anderson would make 1.3 million. Garret Anderson looks pretty similar. So the model isn't over predicting.
I appreciate the SABR community's analysis. If anyone sees any of my papers and has a critique I will try my best to respond.
So consider this an olive branch from an economist who lives in an ivory tower in his mom's basement.
Well, I think the appeal to authority that he, and everybody else, is leery of is the "I'm an expert, end of discussion" type. At least peer-review means that the knowledge claims in question have been vetted. That type of appeal to authority should carry weight.
I was thinking that, but assuming anything from one's screen name seemed a bit risky. In any case, thanks to all for the answer.
Vetted by experts to whom we should defer, he means. And if you think that review committees on scholarly journals aren't affected by who is authoring a paper, you're fooling yourself.
Tango didn't spring fully formed from Bill James's forehead, ready for sabr-types to worship. He has to carefully present and defend his ideas, just like any guy/gal with a bunch of letters after his/her name, and I'd like to think that if he presented bad research, he'd get called on it here.
AFAICT, Bill didn't even use regression models for Runs Created.
A thousand times yes.
For anyone here who has studied labor economics, this style of regression is very common in labor economics. It makes sense to create a logarithmic curve, then use a 'years of experience' variable, add a 'years of education variable' and presto, you have a reasonably accurate model for describing a labor market for, say, economists. Take any number of similar jobs, present them to a bunch of other labor economists, and you'll get a pat on the back and a publication in the journal. Hakes and Sauer, which I wrote my undergrad thesis on, makes this mistake. As far as trying to create a model for baseball salaries in a traditional labor economics style, they do as good a job as they might be expected to. All the while, they're failing to recognize that baseball salaries are really not determined in the same way as salaries for the rest of us.
The problem is that the baseball 'labor market' isn't such at all. It's an auction with a fixed number of lots. There aren't diminishing returns to experience or skill; the marginal price for better players increases as you reach the star players.
He gets called out when he presents good research. Everyone gets called out here.
Right. Using traditional economic models to predict player salaries like trying to use the real estate market in Tempe, AZ to try and predict what it would cost to buy Buckingham Palace. You might come up with a number that's close, but if so, it would be by accident.
Yes, but at least the vetting process is suppose to make a determination of the research's quality as held by scientific standards. That is different from simply stating, "I am the expert" w/o allowing a discussion of the intellectual merits of the research. For example, simply stating "I am the expert" will get you nowhere fast in the peer review process.
And if you think that review committees on scholarly journals aren't affected by who is authoring a paper, you're fooling yourself.
I've served as a deputy editor for a peer reviewed journal. Obviously, in some cases the reviewer knows the author despite it being a double blind process. But there are plenty of instances where both parties don't know the identity of each party.
Edit: Actually, I should say the reviewer can infer the author of the paper despite it being a double-blind process.
My name is Karl. Ich bin expert.
Of course it's Bradbury that's doing this, not the people he criticises. Rather than defend his ideas on their merits, he's going back to first principles and defending those principles with appeals to expertise (I've based this on sound economic principles, which have been vetted by people with expertise, and all this criticism is coming from people who don't know what they're talking about).
When he says that people here are blindly following certain sabrmetric researchers, he's beating up a straw man. Nobody's doing that.
I corrected him slightly:
Kiko/59 is also correct:
He's right. What Adam Jones makes in his 3 pre-arb years is not very related to his performance. And certainly not as much as what he'd make in arb or FA. But that regression equation does not model this reality that everyone here knows.
That Sakes/Hauer paper has many such pitfalls in their model. To then use the year-to-year trend in their equation to say that OBP has gone from being completely (not slightly, but completely) ignored in 2001 to properly valued in 2004 seems to be pretty unreasonable to me.
His skin has been getting thinner over time...
He has his own blog, and has been known to close the comments section (I just checked, commenting is currently open)
He can get very prickly when people question his models, his results or his methods, he used to engage in discussions/debates with his critics, but lately it seems that.. he... doesn't, he just says they're wrong- and not directly to them either.
That's they type of flaw JCB's own models have- they grossly overstate the value of a player who plays everyday as opposed to someone who plays 100, its' because he seems to used -0- as a baseline rather than some replacement level (which he says is a useless concept) granted we could debate all day over what the proper replacement level should be, but any reasonable level is better than saying the guy who drove in 71 runs in 600 ABs is better than the guy who drove in 70 runs in 400 at bats, because he drove in 1 more run and it is speculation that the 400 AB guy would have driven in any more if given the opportunity, and besides if the 400 Ab guy is better why didn't he get 600 at bats...
I am exaggerating a bit...
That's fine. If you notice from my comments above, I'm not particularly sympathetic towards Bradbury in this instance. But I am saying that I give more weight to knowledge claims vetted through a peer review process than other forums whether it's newspapers, blogs, etc. In this particular case with the Hakes and Saucer paper, I'm more likely to trust the peer review process than random reviewers on the Internet (of course, tango is a not a random reviewer on the Internet so I'm certainly willing to lend more gravitas to his opinion).
So far so good; conceptually it made plenty of sense. The problem was that its assessments of performance by players were just ridiculously crude, literally using stats like batting average and pitchers' W-L record. So now matter how valid the economic assumptions/models they used -- and I had no capacity to know -- the fact that they employed the most inexpert of baseball player metrics rendered their work useless. It appeared to be a situation of economists who happened to be casual baseball fans thinking that it would be fun to apply some economic models onto the baseball case: sure, it might have been fun, but unless and until they were willing to subject the baseball side of the project to the same rigor that one hopes they were applying to the economics side, the thing was dead in the water. I've been attending NINE conferences for almost ten years now, and I've rarely witnessed a presentation bomb the way this one did. It was painful.
But I'll agree those results look fishy. About the lowest possible realistic FA-eligible salary -- a starting SS posting a 330/350 line, say -- is estimated at about $3 M. In 2001, this, roughly speaking, would have been Royce Clayton ($4.5 M), Pat Meares ($3.8 M) or Tony Womack ($4 M and not yet an FA) so it's looking pretty badly off at that end. (For 2B, you've got Mike Lansing at $6.25 and McLemore at $2; at 3B, Brosius at $5.25.)
Meanwhile a 400/600 FA-eligible OF is predicted to make less than $6 M. We're talking Barry Bonds ($10.3), Sheffield ($10), Sosa ($12.5), Alou ($5.25), Giles ($7.3), Burks ($5.7), Edmonds ($6.3). So it fits pretty well for 3 of the 8 but is generally way low.
So it's a model which fits the middle of the data but not either end of the data. Pretty much all models do that. The fact that Mike Lansing made more than what the model would project for Gary Sheffield is a pretty good indicator that the model didn't fit the data well.
And I did find this amusing:
a model that the authors are acknowledging is out-of-whack with what should be.
Wasn't it just yesterday that all market decisions were rational? :-)
But why? If you read the paper, and then someone who calls himself Dracula's Garlic on the internet makes a cogent, incisive critique of the paper, what does it matter if it's been professionally vetted or not? Smart is smart.
Right, and you can see that in minor ways (like the clearly incorrect weights for SF and sac bunts)
I'm pretty sure that it started as an accidental discovery -- AB*OBP*SLG being the first version he came out with.
Then a lot of trial and error to get the weights close for stolen bases and then more trial and error for everything else.
Worth noting that there's a little mini-rant in one of the early Abstracts on the subject of regression based formulas by people who don't understand the game. Will see if I can dig it up.
I thing you're misreading him here. He tries to stay within the data to find a 600 PA guy comparable to a 340/400 400 PA guy. But he can't do it. Their model won't admit any player with 600 PA could have as low a salary as the 400 PA 340/400 guy. He only extrapolates to make his point that the model is severely broken.
Make someone a free agent, with 600 PA, a .330 OBP, .400 SLG outfielder. His salary is 3.2MM. Make him a SS or C, and it's 3.4MM.
Does that make sense to anyone?
What if we give him 500 PA instead, with a .500 SLG and .400 OBP. Pretty attractive, right? Well, it's the same 3.2MM$ salary for the OF, and the same 3.4MM$ salary for the SS.
Now, if they are suggesting their empirical model, based on actual performance data of 2000 that says that an OF with 600 PA, a .330 OBP and .400 SLG is making in free agency as much as the SS with 500 PA, with a .400 OBP and .500 SLG, then should I accept that? Do I need to go further to disprove them? Even if they can somehow justify it, isn't it more plausible to think that this is some sort of anomoly whereby the uncertainty in the data is so large as to make such a conclusion insignificant?
You have to have SOME logical underpinnings, don't you? You can't just throw everything into the regression grinder, and think that you will get a filet mignon, or even a kielbasa? Sometimes, you get dog food.
Why is it to me to disprove such an unreasonable claim, and not for the authors of the paper to prove it?
Seems to me that replacement level (as defined by GM choices) ought to be a testable hypothesis. To re-state more meaningfully, GM actions ought to tell us where they see replacement level -- even if I haven't figured out how to run the study.
Nope, Tango.
I'll have to repeat it for the third time: it's not that I tested it only on the extremes. It's that I started at the average, and I started going away from it to see how extreme I needed to go to get an equivalency to a somewhat average player, but with a different playing time.
And, I would expect, in real-life, that I would go down somewhat on the playing time and upwards somewhat on the rate stats to get that equivalency.
It's not that it failed at the extremes, but it failed at every point, that I couldn't even justify it even at the extremes. There was no data point that would make sense.
In other words, the breakdown at the extreme is not the anomoly, but REPRESENTATIVE of what the model was doing. The whole thing made no sense. This is evidenced with my later post (in post 87), and your real-life data (post 83).
Edit: or what snapper/86 said, which is both briefer and clearer. If you follow snapper/86, don't bother reading this post.
But it's peer-reviewed.
Because unless the paper in area that I hold competency, I can't evaluate if a critique is cogent or incisive. I'll defer to the experts rather than simply trust Dracula's Garlic. But like I already said, in this particular case, Dracula's Garlic is a well-known, established researcher himself. I'm certainly willing to trust his evaluations within reasonable limits. Of course, I should just read the damn subset of articles in question in order to come a fully informed opinion. But for now, I'm happy to read the side comments.
I agree, though the work of Talbot's Wolf Bane is much more nuanced, IMO.
But it's peer-reviewed.
Come on, let's not take things to an extreme. Everybody academic has read a peer-reviewed article in flagship journals that puzzled the hell out of them as to why this piece of crap was published.
Check your sarcasm detector.
I can't because I'm getting a lot interference while sitting in my mother's basement.
I know it makes (almost) no difference when modeling team runs scored to use just OBP/SLG, BA/OBP/SLG or BA/IWR/ISO. Not sure why it would matter a lot here.
And OBP*N+SLG (for whatever value of N makes sense in context. N=1.8 is a good choice) isn't a heck of a lot better than simple OPS.
Sensible rounding cures the ills.
As for why SLG seems to have a "wrong" weight in their model, simple explanation for that to my mind. RBI still matter and SLG*(AB with runners on base) has a very high correlation with RBI (particularly in cases headed for an arbitrator. Bill James has said he never breaks out his toolkit in arbitration presentations because it's all on the clock and demonstrating the validity of any method is just too time consuming)
IOW my nit is the absence of rbi from the model.
Another big issue is the failure to deal with salary inflation.
I don't have a philosophical problem with economists studying anything under the sun. I just wish they would use something more sophisticated than regression to do it, or at least go to more pains to make sure that the data set really is linear.
As others have pointed out, regression will happily give you an answer no matter what garbage you put into it. That makes it a dangerous tool, because it won't necessarily warn you that you're putting in garbage. That equation has seven parameters! The coefficient for OBP is negative!
Fermi once said "With four parameters, I can fit an elephant. With five parameters, I can make him wiggle his tail." Seven free parameters will make even crummy models seem to fit the data well.
You must be Registered and Logged In to post comments.
<< Back to main