Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Baseball Newsstand > Baseball Primer Newsblog > Discussion
Baseball Primer Newsblog
— The Best News Links from the Baseball Newsstand

Tuesday, November 20, 2012

Luck and Skill Untangled: The Science of Success | Wired Science | Wired.com

When we post articles which aren’t interesting to you, consider it regression to the mean.

Mauboussin: I think this is a cool analysis. I learned from Tom Tango, a respected sabermetrician, and in statistics it’s called “true score theory.” It can be expressed with a simple equation:

Observed outcome = skill + luck

Here’s the intuition behind it. Say you take a test in math. You’ll get a grade that reflects your true skill — how much of the material you actually know — plus some error that reflects the questions the teacher put on the test. Some days you do better than your skill because the teacher happens to test you only on the material you studied. And some days you do worse than your skill because the teacher happened to include problems you didn’t study. So you grade will reflect your true skill plus some luck.

Of course, we know one of the terms of our equation — the observed outcome — and we can estimate luck.  Estimating luck for a sports team is pretty simple. You assume that each game the team plays is settled by a coin toss. The distribution of win-loss records of the teams in the league follows a binomial distribution. So with these two terms pinned down, we can estimate skill and the relative contribution of skill.

To be more technical, we look at the variance of these terms, but the intuition is that you subtract luck from what happened and are left with skill. This, in turn, lets you assess the relative contribution of the two.

Some aspects of the ranking make sense, and others are not as obvious. For instance, if a game is played one on one, such as tennis, and the match is sufficiently long, you can be pretty sure that the better player will win. As you add players, the role of luck generally rises because the number of interactions rises sharply.

Jim Furtado Posted: November 20, 2012 at 06:47 AM | 5 comment(s) Login to Bookmark
  Tags: sabermetrics

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. depletion Posted: November 20, 2012 at 08:37 AM (#4306599)
The same happens with negative feedback. Should your daughter come home with a poor grade reflecting bad luck, you might chide her and punish her by limiting her time on the computer. Her next test will likely produce a better grade, irrespective of your sermon and punishment.

Of course it's impossible to know if the grade is because of luck so obviously you try to determine why she got the grade: didn't do the homework, test was poorly worded, bad noise/odor in the test room, never really grapsed the material, etc. It's pretty mindless to write off results to "luck" without first seeing if they're "trend".
   2. Rally Posted: November 20, 2012 at 09:53 AM (#4306640)
Depends on the way the test is set up. If it's all multiple choice, with many difficult questions that lead to guessing, then luck is a big factor.

Back in my day, after those of us lucky enough to avoid sabre-tooth tigers made it to school, most of the tests were pretty easy if you read what you were supposed to read and moderately tough if you didn't.
   3. valuearbitrageur Posted: November 20, 2012 at 12:49 PM (#4306814)
In Poker the results = skill + luck works out slightly differently,

because mother ####### donkeys always get way more luck.
   4. Walt Davis Posted: November 20, 2012 at 01:08 PM (#4306836)
Not that it matters much but team outcomes (or batter outcomes) do not strictly meet the assumptions of the binomial distribution. The binomial is (almost certainly) a reasonable approximation. The independence of tests is probably the most "controversial" of the binomial assumptions for baseball (hot and cold streaks).

Also one doesn't normally think of discrete variables in a classical test theory (or "true score theory") way. It doesn't matter much once you're talking outcomes of 162 games or 600 PA but O = T + E is more conformable with continuous variables. For a discrete variable, P(O) = T (not usually expressed as a "true" score but go with it) or in the case of the binomial, E(O) = NT where N is the number of trials, T the true p of success and O the number of successes. As N gets large, the binomial approximates a normal distribution for most baseball-y values of T.
   5. dr. scott Posted: November 20, 2012 at 02:47 PM (#4306912)
Daniel Kahneman descirbes this quite well in thinking fast and Slow when an Israeli general tells him that positive feedback never works and negative feedback always works. when he praised cadets for a clean execution of some maneuver, the next time they try they usually dont do nearly as good, and whever he yelled at someone for screwing up, they always did better.

Kahneman was quite excited by this as he saw this as a perfect example of regression to the mean, but realized that regression ot the mean is one of the main reasons why people think megative feedback is so effective. This of course is most true in cases where you only praise or reprimand serious outliers. Yhere will always be a distribution of talent of course.

You must be Registered and Logged In to post comments.

 

 

<< Back to main

News

All News | Prime News

Old-School Newsstand


BBTF Partner

Support BBTF

donate

Thanks to
Downtown Bookie
for his generous support.

Bookmarks

You must be logged in to view your Bookmarks.

Hot Topics

NewsblogAin't no WEEKEND OMNICHATTER until someone gets punched in the face, for August 19-20, 2017
(96 - 3:10pm, Aug 20)
Last: What did Billy Ripken have against ElRoy Face?

NewsblogOTP 14 August 2014: The American Pastimes of Rock ’n’ Roll, Baseball and Poetry
(2714 - 3:07pm, Aug 20)
Last: Rickey! will gladly sacrifice your janitor

NewsblogArthur: Buster Posey Has Quietly Become A Lock For Cooperstown
(99 - 3:06pm, Aug 20)
Last: cardsfanboy

NewsblogMLB umpire protest: Umps unlikely to get sympathy from fans | SI.com
(7 - 3:03pm, Aug 20)
Last: What did Billy Ripken have against ElRoy Face?

NewsblogJake Arrieta is pitching like an ace again — just in time for free agency - Chicago Tribune
(1 - 2:56pm, Aug 20)
Last: Rough Carrigan

NewsblogJeffrey Loria has ‘no sad feelings whatsoever’ about selling the Marlins - Sun Sentinel
(5 - 2:46pm, Aug 20)
Last: The Duke

NewsblogMLBTR: Dodgers Acquire Curtis Granderson
(19 - 2:45pm, Aug 20)
Last: manchestermets

NewsblogCENTRAL PARK - JULY 1, 2017 10 AM - BBTF ANNUAL SOFTBALL EXTRAVAGANZA
(417 - 2:23pm, Aug 20)
Last: LA Podcasting Hombre of Anaheim

NewsblogTaking Back the Ballparks - Rangers voting thread
(38 - 1:57pm, Aug 20)
Last: SoSHially Unacceptable

NewsblogGregg Olson: Orioles' emphasis on talented, structured bullpens is nothing new - Baltimore Sun
(3 - 1:55pm, Aug 20)
Last: Bote Man

NewsblogOT: New Season August 2017 Soccer Thread
(157 - 1:33pm, Aug 20)
Last: JuanGone..except1game

NewsblogRed Sox want to do away with 'racist' legacy of famed Yawkey Way
(111 - 1:14pm, Aug 20)
Last: ERROR---Jolly Old St. Nick

NewsblogTwins Byron Buxton evolving as a hitter | MLB.com
(26 - 12:58pm, Aug 20)
Last: snapper (history's 42nd greatest monster)

Gonfalon CubsNow or Never
(104 - 12:17pm, Aug 20)
Last: Andere Richtingen

Hall of MeritMost Meritorious Player: 2007 Discussion
(19 - 12:08pm, Aug 20)
Last: Chris Fluit

Page rendered in 0.1537 seconds
47 querie(s) executed