The Signal and the Noise

The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t is Nate Silver’s 2012 meditation on prediction, which investigates how we can distinguish a true signal out of the vast universe of noisy data. Most predictions fail, he asserts, because most people have a poor understanding of uncertainty and probability. Silver visits some of the most successful forecasters, examining predictions of hurricanes and sports as well as poker and the stock market. He observes that the most accurate of forecasters have a superior knowledge of probability, ensuring they know what can and cannot be predicted and with how much confidence, which allows them to distinguish the signal from the noise.

Silver’s method of prediction is very straightforward: he asks how else you would approach a question if not by considering every prior example. He has used it to develop a revolutionary new method for predicting baseball players’ performances, which the website Baseball Prospectus has acquired, and to rake in six figures annually in poker. He was even approached by the Obama campaign for his predictive powers in politics.

The author asserts that the deluge of data in today’s world has made worse the problem of people perceiving patterns where they do not exist and of overfitting predictive models to data of the past. Predictions also frequently fail because they do not take model uncertainty or take an average of different perspectives into account. Silver looks at some specific examples of forecasting to dissect the nature of making those predictions.



When discussing the failure to predict the 2008 housing bubble and subsequent recession, Silver states that people failed to take into account model uncertainty, particularly with the rate of mortgage securities being taken at face value. Additionally, every individual factor that contributed to the housing bubble (perverse incentives, inadequate regulation) seemed like a common occurrence, which would usually be offset by other safeguards, but the crisis seemed to arise from all factors being present at once.

Silver then turns to political predictions, claiming that most pundits do no better than chance when attempting to forecast political events. The experts who tend to do a better job forecasting are usually multidisciplinary, use multiple approaches to forecasting, are willing to change their minds, offer probabilistic predictions, and rely on observation more than theory. Silver integrates historical poll data and information on the economy and the demographics of states to make his political predictions.

Turning to weather, Silver informs us that weather forecasters are able to access a large amount of data, offering them rapid feedback loops that enable them to repeatedly test their hypotheses. Integrated usage of computer models and human judgment does markedly better than computer models alone. He asserts that Hurricane Katrina was not appropriately handled because the local government failed to heed forecasters’ warnings early enough, and the local people failed to take the warnings seriously.



When it comes to earthquakes, Silver explains that the Gutenberg-Richter law predicts their frequency of a given magnitude in a given location, which can be used to predict the frequency of earthquakes of a higher magnitude. However, attempts to build better models that make more precise predictions have been unsuccessful due to an overfitting of existing data.

Silver then focuses on the economy. He claims economists have a poor track record of predicting GDP growth because so much data pertains to factors that might drive it, so it is easy to see patterns that are not real. Furthermore, the economy is very volatile, and past patterns often fail to predict future ones.

When looking at poker, Silver explains that elite poker players use Bayesian reasoning to estimate each hand’s probability based on the cards on the table and contingent upon their opponents’ behavior. They also use additional information, such as the fact that, in general, women play more conservatively than men do, to refine their predictions.



The stock market is the next area Silver focuses on. Certain systematic patterns are recognizable in the stock market, but they are rapidly exploited and disappear when people acknowledge them. A stock market bubble, asserts the author, is not difficult to predict and can be ascertained by looking at the average price to earnings ratio across all stocks, which, when sufficiently high, indicates a bubble. Knowing when a bubble is going to pop, however, is more of a challenge.

Turning to climate change, Silver admits there is much uncertainty regarding climate change predictions, including climate models, initial conditions, and society’s ability to adapt. However, such uncertainty easily justifies focusing on mitigating climate change because the potential risk of the problem being worse than it is expected to be entails much more negative consequences than those in the median case.

Finally, Silver takes on terrorism, stating that while governments often do prepare for attacks, they usually prepare for the incorrect kinds of threats as they are unaware of larger threats. The September 11, 2001, terrorist attacks had not even been considered, he says. There are some reasons, he claims, to be concerned about the possibility of a nuclear weapon or bioterrorism attack in the United States, which could kill over a million people.