I thought the headline said PETCO and was wondering how a simple pet store could ever stir up debate.

Silver astonished well-connected Washington pundits and landed himself a book deal when his political projection site, FiveThirtyEight.com, nailed Barack Obama's victory over John McCain in the presidential election better than any poll could ever dream of.

Actually, pretty much everybody before the election had Obama by 6 or 7 points, 538 wasn't any more accurate than most individual polls or poll aggregation sites. Nate's claim to fame rests on his correctly predicting the results of the primaries between Hillary Clinton and Obama, which most polls got completely wrong, mostly because of the undecided black voters breaking exclusively for Obama. Most of the time a simple poll average would do a good job of predicting election results, so Nate lucked into a situation where his more elaborate approach actually added a lot of value and let him make a name as a poll analyst. Plus Silver is an excellent writer, so that alone makes 538 worth reading.

Also, he got the state breakdown pretty damn good, which is much harder than just calling it for Obama.

Ekogan-

The difference being that Nate's prediction was pretty constant for the last month+ of the election, as well as doing a pretty good job of predicting the Senate and the Democratic party. I think that you are giving luck way to much credit.

Yup. The website was informative, entertaining, well-constructed, thoughtful--just a good job all around. I consult on web design and Nate did particularly smart work in packing huge amounts of information into an accessible format.

It's not that difficult to predict election results using poll aggregation and a bit of political knowledge, it's a lot more impressive to predict election results based on a few economic indicators, like Ray Fair.

It's not that difficult to predict election results using poll aggregation and a bit of political knowledge, it's a lot more impressive to predict election results based on a few economic indicators, like Ray Fair.

What's really impressive is sticking with it, in the face of skeptics who insist that the daily news cycle really matters in election outcomes and that anybody who disagrees must be in denial.

#7: Nate Silver was treated like a superstar by the media during the elections. What's really impressive is sticking with your predictions when a New York Times Magazine interviewer suggests that you are discouraging students from thinking about society in complex and meaningful ways, may be a Republican, or implicitly lending support to Bush, or influencing the public by making predictions, or that forecasts by economists are unimportant.
(link)

Has Silver done any substantive analysis of how well his projection system worked? I don't really think so. Sure, he can point to the electoral vote totals and the national numbers and say, "Look at that. Well done!" But he did miss Indiana & that little district in Nebraska. While that sounds good when you have 50 states (52 including Maine and Nebraska districts?), there were really only 10 or 11 tossups at most. Not to say that silver's model didn't do a good job, I don't really know. (I also don't know how deadly accurate PECOTA is. Just like 538, it seems that most evidence regarding PECOTA's lethality is ancedotal.)

It's unfortunate that Nate Silver hasn't done any real number crunchy sort of things with the electoral results. There's a lot of interesting self-analysis that, while almost assuredly damning 538.com to some sort of backwards looking, statistics ghetto, would make the site more entertaining to me. With the results in hand, we could be getting data on how well the pollsters did. Or whether 538 correctly estimated any bias or consistent error in pollsters. Or, most interestingly to me, how well the regression did compared to the monkey. I'd be more interested in seeing data on whether or the extent to which the regression added any value than in reading about stimulus / bailout / arlen spector. To me, Silver's just another voice on politics / policy.

it's a lot more impressive to predict election results based on a few economic indicators, like Ray Fair

Fair's system is pretty impressive on the simplicity-to-accuracy comparison. If he could have made Introduction to Macroeconomics as simple as predicting presidential elections, yours truly might have taken a different career path.

What's really impressive is sticking with your predictions when a New York Times Magazine interviewer suggests that you are discouraging students from thinking about society in complex and meaningful ways, may be a Republican, or implicitly lending support to Bush, or influencing the public by making predictions, or that forecasts by economists are unimportant.

Deborah Solomon biases are stunning in how flagrant they are. She never challenges subjects she likes/agree with but is all over anyone she disagrees with.

The difference being that Nate's prediction was pretty constant for the last month+ of the election, as well as doing a pretty good job of predicting the Senate and the Democratic party. I think that you are giving luck way to much credit.

So did pollster.com and the Princeton election forecast site. Silver wasn't significantly better than his competition. Of course, all of these sites were more accurate than CNN's or other mainstream media sites for electoral projections, who always go for more drama in their articles and are biased toward categorizing states as toss-ups to avoid making wrong predictions. Silver's advantages over the other polling aggregation sites weren't in better accuracy of his predictions, but in better quality of writing and better frequency of his articles.

It's actually very similar to the baseball forecasting scene. PECOTA, while accurate, is not significantly more accurate than CHONE or ZIPS, while all of these systems are much more accurate than the predictions on ESPN. However, Baseball Prospectus, with its stable of writers, has many more interesting, articulate, original articles than Sean Smith or Dan S. can provide. This makes PECOTA the most popular forecast, rather than its technical merits.

I was a big fan of WILTON. Remember him?

Nate's model ultimately didn't "call" Indiana, but in his final writeup he did say it was the most likely state to buck the model: and it did. He also had Missouri pure white in his last projection, and it ended up taking over a week to call the state.

I don't know why anyone agrees to be interviewed by Solomon. She's pretty sleazy.

Deborah Solomon biases are stunning in how flagrant they are. She never challenges subjects she likes/agree with but is all over anyone she disagrees with.

Uh, unless I'm confusing her with someone else, this is the same Deborah Solomon who actually forges these interviews, right? Somehow, she kept her job despite being exposed as having lied about what was said by the subjects, by changing the questions she asked to make it look like they're saying something entirely different.

Somehow, she kept her job despite being exposed as having lied about what was said by the subjects, by changing the questions she asked to make it look like they're saying something entirely different.

Really? Having read her I would not be surprised that claim is true. Link? How does she have the job at least four years later?

Re: Indiana, I think it's much more impressive that Nate was touting it as a pickup opportunity for Obama as early as April.

In the last couple weeks before the election, there were so many polls that all the aggregator sites were bound to give similar predictions. Also there wasn't any late-breaking news (unless you count Joe the Plumber).

What separated Nate's model from the rest was that he extrapolated to election day rather than taking a "snapshot" of the race at the time. Essentially, he was trying to quantify the chance that a big event would change many voters' minds. This had a lot of value in, say, mid-August. Clearly Obama was ahead. The question was whether he would stay ahead. You can argue that this kind of thing is fundamentally unquantifiable, but at least Nate took the bull by the horns.

I also enjoyed his daily commentary, and the pollster ratings provided food for thought. It sure seems like IVR polling has gotten a bad rap from the rest of the industry. I wouldn't have learned that from pollster.com, Sam Wang, and all the rest.

--

Uh, unless I'm confusing her with someone else, this is the same Deborah Solomon who actually forges these interviews, right? Somehow, she kept her job despite being exposed as having lied about what was said by the subjects, by changing the questions she asked to make it look like they're saying something entirely different.

Really? I don't much like her interviews, but I missed this story. (Edit: looks like I owe a Coke to GGMC.)

We don't go to BP or 538 for the ultimate in accuracy (though 538 was within .1% on the final numbers IIRC). We come for entertainment.

Silver entertains.

Nate's model ultimately didn't "call" Indiana, but in his final writeup he did say it was the most likely state to buck the model: and it did. He also had Missouri pure white in his last projection, and it ended up taking over a week to call the state.

Why should he get points for hedging his bets on Indiana? He's saying: we love the model, the model is great, but it could be wrong on this one really close case. Of course he could have been wrong on that one close case, it was a close case!! (Note that MO, the other close case by his model, was so close he didn't even make a prediction. I believe 538 had NC more comfortably Obama, though it finished closer than IN at O+.33%.) Silver had at most ten or eleven states that he could have potentially gotten wrong and he hedged in the two closest cases by his model. While that was the right move in MO; he got Indiana wrong.

This hedging is not quite as bad as the BPro trick of writing comments that don't match the DTs/PECOTAs so that there's a back cover tout regardless of the outcome, but it still seems to me an opportunity to have it both ways. At least in 538's case, there are some numbers to back it up. Not to say that Silver did a bad job. I'm just curious the extent to which his regression is value added in the prediciton game.

As far as I can tell, 538 didn't beat the polling firm ARG. They had Missouri and Indiana tied while correctly calling all other states (and punting / incorrectly calling NE's 2nd). ARG was also closer on the actual margin of victory (Actual - 7.2%, Silver - 6.1%, ARG - 8%).

What separated Nate's model from the rest was that he extrapolated to election day rather than taking a "snapshot" of the race at the time.

That was just regression to the mean, decreasing over time. Something one could do in one's head.

But Nate gets credit not because he "could have done it" but for actually doing it. And presenting it well. And on the subject of "could have done it," his methodology doesn't look particularly complicated or sophisticated, but the required skills are pretty rare -- I'd guess somewhere between 1 in 100 to 1 in 1000 Americans could actually do the work, and with that level of presentation quality, though ten times as many might be able do a shoddy but superficially similar imitation.

I think he had a .311 lifetime average against the Tigers.(It’s actually .303, but we won’t hold it against him.)

Interesting not that he had the number wrong, but that even this hardcore stat guy still pops off batting average first thing.

Well, he is discussing his memory from growing up, and when he was growing up, I doubt he had a sense of what Pecota's OBP against the Royals was. If he quoted it now, he wouldn't be quoting his memory growing up, but his memory of whenever he last calculated/looked it up.