Nate Silver's Methods Aren't Perfect, But None Are

Statistician Nate Silver is a big part of the reason why election polls are so important in recent elections. His website FiveThirtyEight is one of the most trusted and most cited by spectators and journalists for its thorough use of statistical modeling to predict election outcomes. But not everyone thinks his methods are reliable — Silver was criticized by Huffington Post writer Ryan Grim for allegedly "changing the results of polls." Silver quickly defended himself from the accusations on Twitter and said "every model makes assumptions but we actually test ours based on the evidence," but the doubt cast on his reliability is troubling when it comes this close to an election. Could Nate Silver of FiveThirtyEight be wrong? It's always possible when it comes to the world of data modeling, but he takes great lengths to verify his claims.

Just days before the biggest election in decades, Grim published his article criticizing one of the most trusted sources of political data out there in a very interesting way. Grim didn't accuse Silver of bias per se, but rather said that his methods of analyzing polling data were bringing him undue attention. Silver uses trend-line adjustment, a legitimate statistical technique that's most often used in high-volume data analysis to adjust data for potential sampling errors. Grim, however, has a problem with this method, and implied it was just a way for Silver to make the data seem more suspenseful.

"By the time [Silver's] done adjusting the 'trend line,' Clinton has lost 0.2 points and Trump has gained 1.7 points. An adjustment of below 2 points may not seem like much, but it’s enough to throw off his entire forecast, taking a comfortable 4.6 point Clinton lead and making it look like a nail-biter," Grim wrote.

Silver responded via Twitter just a few hours after the article was published with a pretty thorough explanation of why he does what he does. In a FiveThirtyEight blog post from October that was linked in a tweet, Silver explained that his models take a variety of factors into account that other don't, such as historical data, a higher probability for rare events to occur, and the large number of third-party and undecided voters. Reading through the explainer post gives a pretty good idea of why Silver's model is worthy of trust — he's thought through a lot of possibilities and probabilities to arrive at the basis for his model, and he gives alternative explanations of how different assumptions can change his prediction too.

Ultimately, numbers alone are meaningless until you give them context. What Silver does is apply mathematics to make polling data approximate real life scenarios as closely as possible. And while his methods may not be perfect, that doesn't make them wrong. He makes careful and calculated assumptions to back up his models, which is all one can do when attempting to look into the future. Silver isn't the only, or even necessarily the best, source for election models, but that doesn't make him wrong either. His statistics are sound, and people are still free to check other sources that might give them a more comforting prediction.