Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Baseball Newsstand > Baseball Primer Newsblog > Discussion
Baseball Primer Newsblog
— The Best News Links from the Baseball Newsstand

Thursday, September 29, 2011

The Economist: Never say never

The magazine’s take on the odds of Game 162’s simultaneous comebacks, and why the Rays besting the Red Sox is good for baseball.

There was something on the order of an 0.5% chance the Red Sox would blow their nine-game lead over the Rays, and a 2% chance the Braves would lose their seven-and-a-half-game edge over the Cardinals. Then in tonight’s matchups, there was a 13% chance the Phillies would come back to beat Atlanta, a 5% chance the Orioles would come back to beat Boston, and an 0.3% chance the Rays would come back to beat New York. Multiply it all out, and the odds of witnessing what we just witnessed were worse than one in 500m.

David Concepcion de la Desviacion Estandar (Dan R) Posted: September 29, 2011 at 05:06 PM | 22 comment(s) Login to Bookmark
  Tags: general

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. karlmagnus Posted: September 29, 2011 at 10:47 PM (#3945581)
The Economist clearly has a Wall Street guy writing it sports page. That kind of calculation was the way those bozos did their risk management. None of this stuff was independent; in fact the combined probability of the Sox collapse was in the 2-5% range.
   2. Home Run Teal & Black Black Black Gone! Posted: September 29, 2011 at 10:49 PM (#3945583)
But Sam Hutcheson says nothing of note happened.
   3. David Concepcion de la Desviacion Estandar (Dan R) Posted: September 29, 2011 at 10:55 PM (#3945588)
The author is me, karlmagnus. Why would those events not be independent? How did what happened in, say, Baltimore affect in any way what happened in Philadelphia?
   4. 'zop sympathizes with the wrong ####### people Posted: September 29, 2011 at 10:58 PM (#3945592)
Dan, don't feed the troll.
   5. Athletic Supporter can feel the slow rot Posted: September 29, 2011 at 11:06 PM (#3945597)
The individual games are independent, but you really need to multiply by the probability of teams being tied going into the final day, not by the probability of blowing the lead (which includes final day events), which probably gives you a factor of 3ish. Also, there is a lot of cherrypicking, the old oddmatch fallacy where what happens is very improbable but there were a lot of improbable things that could happen. For instance, you need a factor of 4 because any 3 of the 4 games could have been dramatic. You need a factor of 2 because it could have been the Yankees blowing their lead instead of the Sox, another factor of 2 because it could have been the Tigers blowing a big lead, another factor of 2 because it could have been the Brewers, and so on. Something identical to this happens far more often than 1 in 500m.

Another note: the odds of ever witnessing a true 1 in 500m event (i.e. something that really is unique and not just an example of a larger class) are basically zero. If you come up with the probability of something that actually happened as 1 in 500m, at worst you are falling into the statistical fallacy above (the "someone has to win the lottery" fallacy, e.g. how unlikely is it that Barack Obama is president? There are 200M eligible Americans, that's a 1 in 200 million event!), and more likely your inputs or thought process are incorrect.

This article and Silver's are both disingenuous in their use of numbers, which makes me sad -- Dan, obviously you are a smart guy and Nate is a smart guy, but this sensationalist use of numbers is just not necessary.
   6. karlmagnus Posted: September 29, 2011 at 11:07 PM (#3945599)
Not a troll for once; I've published a book on the Wall Street version of this "Alchemists of Loss" Kevin Dowd and Martin Hutchinson, Wiley 2010. Baltimore was certainly NOT independent of the overall collapse, which required a series of bad luck events like Baltimore in order to happen. Also, the probability of collapse was never anything like as low as 0.5% because that failed to take account of the team changing in quality in September through injuries and a string of losses -- error rate tripling, for example. Most though not all of the things that happened to the Sox in September were deterministic, not random.

Clearly the AL result was nearly independent of the NL result, with only a mild psychological overlap between the two in the last few days.
   7. I Am Not a Number Posted: September 29, 2011 at 11:28 PM (#3945607)
How did what happened in, say, Baltimore affect in any way what happened in Philadelphia?

Butterfly Effect.
   8. danup Posted: September 29, 2011 at 11:43 PM (#3945613)
Sir,

Don't you mean "the newspaper’s take on the odds of Game 162’s simultaneous comebacks"? I think this is a fireable offense at the Economist.
   9. BDC Posted: September 30, 2011 at 12:02 AM (#3945619)
I also admire your work, Dan, but I think that Athletic Supporter is right. In the last week, Craig Gentry hit an IPHR as his first major-league homer, while the Rays pulled off a round-the-horn triple play and the Diamondbacks came back from a five-run deficit with two out in their last inning. Multiply that by the chance that the White Sox would trade their manager to Florida and that I would rent the Lon Chaney version of The Hunchback of Notre Dame out of the hundreds of thousands of DVDs in print, and basically last week couldn't have happened even before you add in last night :)

Wild finishes have happened in baseball before, and this one, while extremely wild, was not different in kind or even that much in degree. It certainly isn't a 1-in-500M shot: what does the 500M mean in that instance? That it would happen every 500M seasons, or that this precise combination of scores and events would happen as opposed to the near-infinite number of other possible scores and events? If the former, that just doesn't match observation, and if the latter, well, the same is true of every precise combination, as Supporter notes.
   10. JRVJ Posted: September 30, 2011 at 02:23 AM (#3945691)
Nonetheless, even if Tampa Bay’s comeback will cost Major League Baseball money in the short run, the club’s success has positive ripple effects. Sports leagues need their big-market teams to win more than their fair share of the time, but not so often as to remove all suspense about the outcome. The presence of franchises with no hope whatsoever hurts every club—even rich ones, whose fans may not want to pay to see their squad obliterate an overmatched rival. The fact that the Rays have made the playoffs three times in four years gives the lie to the moans of other small-market teams that they cannot afford to compete. Tampa Bay’s victory is a win for fans in Pittsburgh, Kansas City and San Diego as well.


If I may enquire, was the above final paragraph Dan R's writing or was it influenced or massaged by the editors of the Economist. I ask because you can agree or disagree with the ideas espoused here, but what is pretty clear is that this reads like Economist 101 writing (or as Matt Yglesias has written, the EConomist almost goes rote when dealing with a country's problems, whatever they may be, because it almost immediately espouses that said country should make its labor markets more flexible).
   11. David Concepcion de la Desviacion Estandar (Dan R) Posted: September 30, 2011 at 02:49 AM (#3945708)
Athletic Supporter and Bob Dernier Cri--your point is very well-taken. To be perfectly honest, I was racing to get something up before going to bed, and didn't have the time or data to do more than a back-of-the-envelope calculation. I will post an addendum to the blog recognizing that any permutation is of course unique and in and of itself highly improbable. What the "right" number for the odds is, though, depends on what we're asking. Is it, what was the likelihood that any two teams would blow 9-game leads times the likelihood we'd see at least 3 comebacks from <15% win probability on the last day of the season? Even then, that's still cherrypicking/selective endpointing...What I do believe to be true, unless someone can show me a counterexample, is that this was the least probable end to a regular season in MLB history.

danup--I don't think we're required to keep house style outside of our own Web site!

JRVJ--that was indeed my writing, unedited. Could you elaborate on what you mean by it "reads like Economist 101 writing?" I would like to think we recommend policies tailored to wherever and whatever we're writing about. But since we believe that most countries have too many restrictions on their labor markets, it is indeed often the case that among our suggestions would be to remove some of them. I would call that consistency.
   12. JRVJ Posted: September 30, 2011 at 03:06 AM (#3945724)
Dan,

I started reading the Economist circa 1993 or 1994, and was absolutely astonished by the depth of the Economist's pieces, especially its surveys. I was also very pleasantly surprised at how sensible the writing was (to give you an example of one of my first issues, which was a survey on Islam or Islam and Christianity from 1994ish, I remember a very nuanced piece on Islamic banking and how "interest" had been dealt with via financial instruments that seemed to come close to but didn't breach the letter of the Koran).

In the last 5 to 7 years, the Economist seems to have ossified or at least fallen into group think on some issues. I seem to recall a goodbye piece in one of the X-Mas issues from a former Economist female correspondent who warned of this trend.

Some years ago (sometime between the mid 00s and the fall of Lehman Brothers) I was very impressed by piece in the Economist about how what countries should worry about is not inequality, but poverty, since worrying about inequality sometimes entailed chasing a chimera (or at the very least, pushed countries into finding solutions to the wrong problem). While I fully bought that rationale for years, as time has passed, this rationale has almost become an end in itself (at least as espoused by the Economist), without the nuance necessary to realize that while poverty is indeed the real problem, too much inequality can absolutely poison the public discourse and preclude combating poverty (arguably, defending the economic system that begat inequality becomes an end in itself).

An example of my qualms can be seen in a very recent Lexington column about class warfare and the U.S., where the writer of the piece bent over backwards to make a point about how Americans really shouldn't worry about the fact that the rich pay fewer taxes than anybody else in the U.S. and that any suggestion to the contrary is malarkey.

Getting back to your question (and sorry for my rambling), I read the final line in your blog post ("The fact that the Rays have made the playoffs three times in four years gives the lie to the moans of other small-market teams that they cannot afford to compete. Tampa Bay’s victory is a win for fans in Pittsburgh, Kansas City and San Diego as well") as part of a general Economist mindset which I am finding some fault with.
   13. David Concepcion de la Desviacion Estandar (Dan R) Posted: September 30, 2011 at 03:30 AM (#3945740)
I've updated the blog to address the criticism from Athletic Supporter and Bob Dernier Cri (and quoted you, Bob).

JRVJ--Well, on the broader issue of competitive balance in baseball, I think both that revenue disparities are a gigantic problem and threat to the league's long-run health, and that "welfare queen" franchises pocketing revenue-sharing dollars and not investing in the product on the field are poxes on the sport. As I've written in my New York Times column, my solution to the issue (which would of course be vetoed by the big-market clubs) would be to abolish both the draft and revenue sharing as we know it, and replace them with a transfer system based on market size. The Yankees would just have to write $10 million checks to the Royals, Brewers etc. every year no matter how much revenue the clubs happened to generated in a given season.
   14. Don Malcolm Posted: September 30, 2011 at 03:32 AM (#3945741)
All of this (specifically posts 5, 6, 9) references a new type of problem that comes into play in a world where "instant journalism" has been anointed as preferable to a more measured approach. The race to get content into play jeopardizes the quality of that content, and while there are distinct advantages to having a forum such as this, where a cadre of "semi-professional critics" can perform an ex-post facto editing service, one thinks it might be appropriate to create a BTF editor's fund that the writer-for-$$ folks who post here can pay into in order to at least compensate these folks for performing services that aren't being supplied by the writer's actual employers!

As I suggested elsewhere, it's the scenario of collapse that is most striking here, and is qualitatively different in the league structure that we currently have (about to be messed with and most likely in a detrimental way) than would have been the case in the pre-expansion days. The closest analogue would be pennant races where a team had a big lead fairly late in the year and found itself slowly-but-suddenly in a three- or four-way race at season's end.

There were three possible outcomes for each league last night--team A (collapsing team)wins, team B (chasing team) loses; team A loses, team B wins; both teams win/lose and they go to a one-game playoff, with the next outcome. Do those scenarios have any relevance in this discussion at all? How likely was it that we got the exact outcome we got as opposed to a mixed result?
   15. Jolly Old St. Nick Is A Jolly Old St. Crip Posted: September 30, 2011 at 03:45 AM (#3945751)
AFAICT, the combined odds of the Phillies coming back in the ninth inning (13%) x the odds of the Orioles coming back with 2 outs in the ninth (5%) x the odds of the Rays coming back from 7-0 in the 8th inning (1 in 333) would be roughly .00002, or 1 in 50,000.

But of course since one of the other conditions is that this sequence of games has to take place on the 162nd day of the season in order to duplicate last night's postseason-deciding ending, that makes it more like 1 in 8,100,000 that we'll ever see anything like this again. And this has nothing to do with any preceding events that got those 3 teams (the Braves, the Sox and the Rays) up to that point.

Not being a mathematician, I'm sure I'm missing something here, but if so, I don't know what it is. I took all the numbers in the first paragraph from the Economist article.
   16. Slivers of Maranville descends into chaos (SdeB) Posted: September 30, 2011 at 04:29 AM (#3945777)

Another note: the odds of ever witnessing a true 1 in 500m event (i.e. something that really is unique and not just an example of a larger class) are basically zero.


Not since the invention of Youtube.
   17. Russ Posted: September 30, 2011 at 11:20 AM (#3945850)

There were three possible outcomes for each league last night--team A (collapsing team)wins, team B (chasing team) loses; team A loses, team B wins; both teams win/lose and they go to a one-game playoff, with the next outcome. Do those scenarios have any relevance in this discussion at all? How likely was it that we got the exact outcome we got as opposed to a mixed result?


Fangraphs got it right.... the problem with saying the "Probability of X" is that it is really "Probability of X at Time t". You can't arbitrarily pick a time point and then say the probability of the event happening is 1 in whatever because of what AS said in 5. The most interesting way to view the events of that last day is to ask the question: "Given everything that had happened up to time t<T, what was the probability of would happen at final time T?" As you could see from the graph, they started the night about 50/50 to win the Wild Card because conditional on getting to a situation like last night, they were about 50/50. What happened over the course of the night that was pretty extraordinary was the speed at which the tables completely turned. Over 12 minutes, you went from near certainty of the Red Sox going to the playoffs (try flipping a coin 4 times and seeing if you get EXACTLY 7 heads -- it will happen about 11% of the time which, although not impossible, is really not that often) to certainty of them not being in the playoffs. Even when it comes down to a single game (say, a 5 run rally in the bottom of the ninth of a Game 163), it rarely happens over such a short interval. Pos nailed the reasons for the specialness of the moment in his column... it is exactly because nothing like this ever happens so quickly in baseball that made the event so extraordinarily rare, not the existence of the event itself.


If you want to play the game that Dan and Nate played, you have to define your events properly (viewing this strictly from a Red Sox fan viewpoint, which seems to be de rigueur for those RS fans anyways):

Find times t_1 and t_2 such that:

p1 = Pr(Red Sox having an X game lead over the next closest wild card competitor at time t_1)

p2 = Pr(Any team having narrowed the X game lead to a tie at time t=Game 162 - epsilon | Red Sox having an X game lead over the next closest wild card competitor at time t_1) : This is important because you've probably chosen t_1 so that the RS had the biggest lead, so regression to the mean says that this probability will be larger than you would expect

p3 = Pr(Getting to the game situation of interest on The Night at time t_2 | Any team having narrowed the X game lead to a tie at time t=Game 162 - epsilon & Red Sox having an X game lead over the next closest wild card competitor at time t_1)

p4 = Pr(Tamp winning the wild card | Getting to the game situation of interest on The Night at time t_2 & Any team having narrowed the X game lead to a tie at time t=Game 162 - epsilon & Red Sox having an X game lead over the next closest wild card competitor at time t_1)

Then you can take p=p1*p2*p3*p4, but recognize that p is really a function of t_1 and t_2 (say p(t1_, t_2)). Both of the authors have fallen to the fallacy of choosing t_1 and t_2 so that p(t_1, t_2) is as small as possible, when it would be much more interesting to know the LARGEST value possible that this could be.

And as AS points out, this only tells you the probability of the INTERSECTION of those particular events.... you'd have to do a lot of marginalization to get the probability of some event marginalized over t_1 and t_2.
   18. AndrewJ Posted: September 30, 2011 at 11:54 AM (#3945856)
It's like the JFK assassination conspiracy theorists claiming the odds were "100,000 trillion to one" that X number of people connected to the assassination would mysteriously die so soon after November 22, 1963. As Vincent Bugliosi pointed out in Reclaiming History, the odds were a lot less than that (especially when you clarify the terms "connected to the assassination," "mysteriously die" and "so soon after November 22, 1963")...
   19. cercopithecus aethiops Posted: September 30, 2011 at 12:47 PM (#3945887)
the odds of ever witnessing a true 1 in 500m event (i.e. something that really is unique and not just an example of a larger class) are basically zero.


Each unique individual human being presently alive is about a 1 in 7b event. The notion that just because it happened it must not be all that rare is also a fallacy.
   20. Der-K and the statistical werewolves. Posted: September 30, 2011 at 01:11 PM (#3945905)
Your odds of being a specific living human (out of all humans) are 1 in 7b. The odds of being currently alive, given that you're a human, are (probably) about one in eight and - if you've read this sentence - very, very close to unity.

Anyway, I'm with Russ.
   21. Swedish Chef Posted: September 30, 2011 at 01:18 PM (#3945911)
Another note: the odds of ever witnessing a true 1 in 500m event (i.e. something that really is unique and not just an example of a larger class) are basically zero.

There are 86400 seconds in a day, humans can perceive something like 10 things in a second. A true 1 in 500m event should be roughly a once in a couple years occurrence. The odds that you are aware that you're experiencing something utterly rare is probably on the order of 1 in 500m.
   22. BDC Posted: September 30, 2011 at 01:34 PM (#3945923)
Dan, you are extremely gracious, especially given that you know about 500M times more about statistics than I do. Cheers!

You must be Registered and Logged In to post comments.

 

 

<< Back to main

BBTF Partner

Support BBTF

donate

Thanks to
Backlasher
for his generous support.

Bookmarks

You must be logged in to view your Bookmarks.

Hot Topics

Newsblog2014 WORLD SERIES GAME 2 OMNICHATTER
(306 - 10:24pm, Oct 22)
Last: AT-AT at bat@AT&T

NewsblogJerome Williams re-signs with Phils
(9 - 10:21pm, Oct 22)
Last: stevegamer

NewsblogStatcast: Posey out at the plate
(13 - 10:21pm, Oct 22)
Last: Walt Davis

NewsblogCardinals proud of fourth straight NLCS appearance | cardinals.com
(62 - 10:20pm, Oct 22)
Last: cardsfanboy

NewsblogMcSweeneys: NEW BASEBALL STATISTICS.
(2 - 10:18pm, Oct 22)
Last: Depressoteric feels Royally blue these days

NewsblogOT: Politics, October 2014: Sunshine, Baseball, and Etch A Sketch: How Politicians Use Analogies
(3182 - 10:05pm, Oct 22)
Last: The Id of SugarBear Blanks

NewsblogJay set for surgery — and for CF in 2015 : Sports
(5 - 9:58pm, Oct 22)
Last: cardsfanboy

NewsblogOT: Monthly NBA Thread - October 2014
(341 - 9:56pm, Oct 22)
Last: Norcan

NewsblogAd Week: What Is Madeleine Albright Doing on the Wheaties Box?
(1 - 9:54pm, Oct 22)
Last: Howie Menckel

NewsblogHow Wall Street Strangled the Life out of Sabermetrics | VICE Sports
(7 - 9:47pm, Oct 22)
Last: Arbitol Dijaler

NewsblogSielski: A friend fights for ex-Phillie Dick Allen's Hall of Fame induction
(180 - 9:47pm, Oct 22)
Last: AROM

NewsblogHunter Pence responds to Royals fan signs with monster Game 1 | MLB.com
(54 - 7:50pm, Oct 22)
Last: JE (Jason)

NewsblogMike Scioscia, Matt Williams voted top managers
(43 - 7:45pm, Oct 22)
Last: catomi01

NewsblogRoyals are not the future of baseball | FOX Sports
(28 - 7:11pm, Oct 22)
Last: Belfry Bob

NewsblogDealing or dueling – what’s a manager to do? | MGL on Baseball
(39 - 7:10pm, Oct 22)
Last: bobm

Page rendered in 0.4462 seconds
53 querie(s) executed