The Economist clearly has a Wall Street guy writing it sports page. That kind of calculation was the way those bozos did their risk management. None of this stuff was independent; in fact the combined probability of the Sox collapse was in the 2-5% range.

But Sam Hutcheson says nothing of note happened.

The author is me, karlmagnus. Why would those events not be independent? How did what happened in, say, Baltimore affect in any way what happened in Philadelphia?

Dan, don't feed the troll.

The individual games are independent, but you really need to multiply by the probability of teams being tied going into the final day, not by the probability of blowing the lead (which includes final day events), which probably gives you a factor of 3ish. Also, there is a lot of cherrypicking, the old oddmatch fallacy where what happens is very improbable but there were a lot of improbable things that could happen. For instance, you need a factor of 4 because any 3 of the 4 games could have been dramatic. You need a factor of 2 because it could have been the Yankees blowing their lead instead of the Sox, another factor of 2 because it could have been the Tigers blowing a big lead, another factor of 2 because it could have been the Brewers, and so on. Something identical to this happens far more often than 1 in 500m.

Another note: the odds of ever witnessing a true 1 in 500m event (i.e. something that really is unique and not just an example of a larger class) are basically zero. If you come up with the probability of something that actually happened as 1 in 500m, at worst you are falling into the statistical fallacy above (the "someone has to win the lottery" fallacy, e.g. how unlikely is it that Barack Obama is president? There are 200M eligible Americans, that's a 1 in 200 million event!), and more likely your inputs or thought process are incorrect.

This article and Silver's are both disingenuous in their use of numbers, which makes me sad -- Dan, obviously you are a smart guy and Nate is a smart guy, but this sensationalist use of numbers is just not necessary.

Not a troll for once; I've published a book on the Wall Street version of this "Alchemists of Loss" Kevin Dowd and Martin Hutchinson, Wiley 2010. Baltimore was certainly NOT independent of the overall collapse, which required a series of bad luck events like Baltimore in order to happen. Also, the probability of collapse was never anything like as low as 0.5% because that failed to take account of the team changing in quality in September through injuries and a string of losses -- error rate tripling, for example. Most though not all of the things that happened to the Sox in September were deterministic, not random.

Clearly the AL result was nearly independent of the NL result, with only a mild psychological overlap between the two in the last few days.

How did what happened in, say, Baltimore affect in any way what happened in Philadelphia?

Butterfly Effect.

Sir,

Don't you mean "the newspaper’s take on the odds of Game 162’s simultaneous comebacks"? I think this is a fireable offense at the Economist.

I also admire your work, Dan, but I think that Athletic Supporter is right. In the last week, Craig Gentry hit an IPHR as his first major-league homer, while the Rays pulled off a round-the-horn triple play and the Diamondbacks came back from a five-run deficit with two out in their last inning. Multiply that by the chance that the White Sox would trade their manager to Florida and that I would rent the Lon Chaney version of The Hunchback of Notre Dame out of the hundreds of thousands of DVDs in print, and basically last week couldn't have happened even before you add in last night :)

Wild finishes have happened in baseball before, and this one, while extremely wild, was not different in kind or even that much in degree. It certainly isn't a 1-in-500M shot: what does the 500M mean in that instance? That it would happen every 500M seasons, or that this precise combination of scores and events would happen as opposed to the near-infinite number of other possible scores and events? If the former, that just doesn't match observation, and if the latter, well, the same is true of every precise combination, as Supporter notes.

Nonetheless, even if Tampa Bay’s comeback will cost Major League Baseball money in the short run, the club’s success has positive ripple effects. Sports leagues need their big-market teams to win more than their fair share of the time, but not so often as to remove all suspense about the outcome. The presence of franchises with no hope whatsoever hurts every club—even rich ones, whose fans may not want to pay to see their squad obliterate an overmatched rival. The fact that the Rays have made the playoffs three times in four years gives the lie to the moans of other small-market teams that they cannot afford to compete. Tampa Bay’s victory is a win for fans in Pittsburgh, Kansas City and San Diego as well.

If I may enquire, was the above final paragraph Dan R's writing or was it influenced or massaged by the editors of the Economist. I ask because you can agree or disagree with the ideas espoused here, but what is pretty clear is that this reads like Economist 101 writing (or as Matt Yglesias has written, the EConomist almost goes rote when dealing with a country's problems, whatever they may be, because it almost immediately espouses that said country should make its labor markets more flexible).

Athletic Supporter and Bob Dernier Cri--your point is very well-taken. To be perfectly honest, I was racing to get something up before going to bed, and didn't have the time or data to do more than a back-of-the-envelope calculation. I will post an addendum to the blog recognizing that any permutation is of course unique and in and of itself highly improbable. What the "right" number for the odds is, though, depends on what we're asking. Is it, what was the likelihood that any two teams would blow 9-game leads times the likelihood we'd see at least 3 comebacks from <15% win probability on the last day of the season? Even then, that's still cherrypicking/selective endpointing...What I do believe to be true, unless someone can show me a counterexample, is that this was the least probable end to a regular season in MLB history.

danup--I don't think we're required to keep house style outside of our own Web site!

JRVJ--that was indeed my writing, unedited. Could you elaborate on what you mean by it "reads like Economist 101 writing?" I would like to think we recommend policies tailored to wherever and whatever we're writing about. But since we believe that most countries have too many restrictions on their labor markets, it is indeed often the case that among our suggestions would be to remove some of them. I would call that consistency.

Dan,

I started reading the Economist circa 1993 or 1994, and was absolutely astonished by the depth of the Economist's pieces, especially its surveys. I was also very pleasantly surprised at how sensible the writing was (to give you an example of one of my first issues, which was a survey on Islam or Islam and Christianity from 1994ish, I remember a very nuanced piece on Islamic banking and how "interest" had been dealt with via financial instruments that seemed to come close to but didn't breach the letter of the Koran).

In the last 5 to 7 years, the Economist seems to have ossified or at least fallen into group think on some issues. I seem to recall a goodbye piece in one of the X-Mas issues from a former Economist female correspondent who warned of this trend.

Some years ago (sometime between the mid 00s and the fall of Lehman Brothers) I was very impressed by piece in the Economist about how what countries should worry about is not inequality, but poverty, since worrying about inequality sometimes entailed chasing a chimera (or at the very least, pushed countries into finding solutions to the wrong problem). While I fully bought that rationale for years, as time has passed, this rationale has almost become an end in itself (at least as espoused by the Economist), without the nuance necessary to realize that while poverty is indeed the real problem, too much inequality can absolutely poison the public discourse and preclude combating poverty (arguably, defending the economic system that begat inequality becomes an end in itself).

An example of my qualms can be seen in a very recent Lexington column about class warfare and the U.S., where the writer of the piece bent over backwards to make a point about how Americans really shouldn't worry about the fact that the rich pay fewer taxes than anybody else in the U.S. and that any suggestion to the contrary is malarkey.

Getting back to your question (and sorry for my rambling), I read the final line in your blog post ("The fact that the Rays have made the playoffs three times in four years gives the lie to the moans of other small-market teams that they cannot afford to compete. Tampa Bay’s victory is a win for fans in Pittsburgh, Kansas City and San Diego as well") as part of a general Economist mindset which I am finding some fault with.

I've updated the blog to address the criticism from Athletic Supporter and Bob Dernier Cri (and quoted you, Bob).

JRVJ--Well, on the broader issue of competitive balance in baseball, I think both that revenue disparities are a gigantic problem and threat to the league's long-run health, and that "welfare queen" franchises pocketing revenue-sharing dollars and not investing in the product on the field are poxes on the sport. As I've written in my New York Times column, my solution to the issue (which would of course be vetoed by the big-market clubs) would be to abolish both the draft and revenue sharing as we know it, and replace them with a transfer system based on market size. The Yankees would just have to write $10 million checks to the Royals, Brewers etc. every year no matter how much revenue the clubs happened to generated in a given season.

All of this (specifically posts 5, 6, 9) references a new type of problem that comes into play in a world where "instant journalism" has been anointed as preferable to a more measured approach. The race to get content into play jeopardizes the quality of that content, and while there are distinct advantages to having a forum such as this, where a cadre of "semi-professional critics" can perform an ex-post facto editing service, one thinks it might be appropriate to create a BTF editor's fund that the writer-for-$$ folks who post here can pay into in order to at least compensate these folks for performing services that aren't being supplied by the writer's actual employers!

As I suggested elsewhere, it's the scenario of collapse that is most striking here, and is qualitatively different in the league structure that we currently have (about to be messed with and most likely in a detrimental way) than would have been the case in the pre-expansion days. The closest analogue would be pennant races where a team had a big lead fairly late in the year and found itself slowly-but-suddenly in a three- or four-way race at season's end.

There were three possible outcomes for each league last night--team A (collapsing team)wins, team B (chasing team) loses; team A loses, team B wins; both teams win/lose and they go to a one-game playoff, with the next outcome. Do those scenarios have any relevance in this discussion at all? How likely was it that we got the exact outcome we got as opposed to a mixed result?

AFAICT, the combined odds of the Phillies coming back in the ninth inning (13%) x the odds of the Orioles coming back with 2 outs in the ninth (5%) x the odds of the Rays coming back from 7-0 in the 8th inning (1 in 333) would be roughly .00002, or 1 in 50,000.

But of course since one of the other conditions is that this sequence of games has to take place on the 162nd day of the season in order to duplicate last night's postseason-deciding ending, that makes it more like 1 in 8,100,000 that we'll ever see anything like this again. And this has nothing to do with any preceding events that got those 3 teams (the Braves, the Sox and the Rays) up to that point.

Not being a mathematician, I'm sure I'm missing something here, but if so, I don't know what it is. I took all the numbers in the first paragraph from the Economist article.

Another note: the odds of ever witnessing a true 1 in 500m event (i.e. something that really is unique and not just an example of a larger class) are basically zero.

Not since the invention of Youtube.

There were three possible outcomes for each league last night--team A (collapsing team)wins, team B (chasing team) loses; team A loses, team B wins; both teams win/lose and they go to a one-game playoff, with the next outcome. Do those scenarios have any relevance in this discussion at all? How likely was it that we got the exact outcome we got as opposed to a mixed result?

Fangraphs got it right.... the problem with saying the "Probability of X" is that it is really "Probability of X at Time t". You can't arbitrarily pick a time point and then say the probability of the event happening is 1 in whatever because of what AS said in 5. The most interesting way to view the events of that last day is to ask the question: "Given everything that had happened up to time t<T, what was the probability of would happen at final time T?" As you could see from the graph, they started the night about 50/50 to win the Wild Card because conditional on getting to a situation like last night, they were about 50/50. What happened over the course of the night that was pretty extraordinary was the speed at which the tables completely turned. Over 12 minutes, you went from near certainty of the Red Sox going to the playoffs (try flipping a coin 4 times and seeing if you get EXACTLY 7 heads -- it will happen about 11% of the time which, although not impossible, is really not that often) to certainty of them not being in the playoffs. Even when it comes down to a single game (say, a 5 run rally in the bottom of the ninth of a Game 163), it rarely happens over such a short interval. Pos nailed the reasons for the specialness of the moment in his column... it is exactly because nothing like this ever happens so quickly in baseball that made the event so extraordinarily rare, not the existence of the event itself.


If you want to play the game that Dan and Nate played, you have to define your events properly (viewing this strictly from a Red Sox fan viewpoint, which seems to be de rigueur for those RS fans anyways):

Find times t_1 and t_2 such that:

p1 = Pr(Red Sox having an X game lead over the next closest wild card competitor at time t_1)

p2 = Pr(Any team having narrowed the X game lead to a tie at time t=Game 162 - epsilon | Red Sox having an X game lead over the next closest wild card competitor at time t_1) : This is important because you've probably chosen t_1 so that the RS had the biggest lead, so regression to the mean says that this probability will be larger than you would expect

p3 = Pr(Getting to the game situation of interest on The Night at time t_2 | Any team having narrowed the X game lead to a tie at time t=Game 162 - epsilon & Red Sox having an X game lead over the next closest wild card competitor at time t_1)

p4 = Pr(Tamp winning the wild card | Getting to the game situation of interest on The Night at time t_2 & Any team having narrowed the X game lead to a tie at time t=Game 162 - epsilon & Red Sox having an X game lead over the next closest wild card competitor at time t_1)

Then you can take p=p1*p2*p3*p4, but recognize that p is really a function of t_1 and t_2 (say p(t1_, t_2)). Both of the authors have fallen to the fallacy of choosing t_1 and t_2 so that p(t_1, t_2) is as small as possible, when it would be much more interesting to know the LARGEST value possible that this could be.

And as AS points out, this only tells you the probability of the INTERSECTION of those particular events.... you'd have to do a lot of marginalization to get the probability of some event marginalized over t_1 and t_2.

It's like the JFK assassination conspiracy theorists claiming the odds were "100,000 trillion to one" that X number of people connected to the assassination would mysteriously die so soon after November 22, 1963. As Vincent Bugliosi pointed out in Reclaiming History, the odds were a lot less than that (especially when you clarify the terms "connected to the assassination," "mysteriously die" and "so soon after November 22, 1963")...

the odds of ever witnessing a true 1 in 500m event (i.e. something that really is unique and not just an example of a larger class) are basically zero.

Each unique individual human being presently alive is about a 1 in 7b event. The notion that just because it happened it must not be all that rare is also a fallacy.

Your odds of being a specific living human (out of all humans) are 1 in 7b. The odds of being currently alive, given that you're a human, are (probably) about one in eight and - if you've read this sentence - very, very close to unity.

Anyway, I'm with Russ.

Another note: the odds of ever witnessing a true 1 in 500m event (i.e. something that really is unique and not just an example of a larger class) are basically zero.

There are 86400 seconds in a day, humans can perceive something like 10 things in a second. A true 1 in 500m event should be roughly a once in a couple years occurrence. The odds that you are aware that you're experiencing something utterly rare is probably on the order of 1 in 500m.

Dan, you are extremely gracious, especially given that you know about 500M times more about statistics than I do. Cheers!