A Review of Nate Silver's The Signal and the Noise: A Love Story

I’m not sure why I clicked “purchase” on my kindle. But I did. It was late. 1:00 am. Maybe 1:45, I don’t recall. My girlfriend slept next to my while I dicked around online looking for a new book to read. Warren Beatty’s Reds played a whisper above mute on the TV. Maybe John Reed’s Ten Days that Shook the World. The edition with an introduction by Lenin cost $8.95. But Beatty’s film probably covered the major plot points. Amazon recommended Trotsky’s The History of the Russian Revolution, though it was almost $30 in e-book form. And anyway, I could probably find a pdf of either online.

Twitter intervened. I floated from article to article. I clicked on a Breitbart story about the supposed flack ESPN insiders have thrown at Nate Silver’s 538 site. Low hit count. Little ad revenue. This reminded me that his site exists. I started reading an article about burrito rankings but didn’t finish. I googled his name. Luddite journalists write some pretty stupid shit about Nate Silver.  

At this point in the night I felt sleepy. Maybe it was time to put the kindle down. But I then remembered that Nate Silver had written a book a few years back. I found it on Amazon. It’s called The Signal and the Noise. Almost Faulknerian. My girlfriend ruffled the sheets in unconscious agitation before settling back into her deep slumber. I summarily purchased the book and began to read it.

The book has a hazy, dreamlike quality. In it Nate Silver is a journalist who travels around the country talking to experts from the fields of finance, psychology, meteorology, seismology, and computer science about why their colleagues aren’t very good at forecasting. These interviews typically serve as a framing device for the whole chapter. Each chapter covers the forecasting deficiencies found in a different field. Nate Silver then offers his own advice to the struggling natural or social scientist: think more probabilistically. Be a fox. Sniff out the truth (more on this business later).  

There are a few deviations from this formula. In one chapter Silver approves of recent advances in weather forecasting, though he laments the fact that economic incentives sometimes lead companies like the Weather Channel to fudge the facts. He notices the same dynamic when diagnosing why financial models failed to predict the 2008 crisis. Nevertheless, elsewhere in the book he firmly believes that competitive markets incentivize better forecasting. This seeming inconsistency may in fact belie Silver’s more nuanced understanding of the tensions inherent within any complex system, particularly capitalist markets, but at no point does he actually address this. So instead we’re left with markets yield the truth, accept when they don’t.

Another deviation from the book’s formula comes in the chapter about baseball. In baseball we have good forecasting because Nate Silver invented a good forecasting model. This is more or less true. Silver’s PECOTA improved upon older quant approaches in rather impressive ways. I would’ve liked Silver to have gone into more detail discussing the particulars of his model, but I sense that because it’s a popular trade publication Silver (or maybe his editor) doesn’t think his audience would understand it. Also, I doubt he’d want to reveal his special sauce because, well, capitalism.  

Early on in the book Silver morphs from a journalist into a fox. The transition is subtle, yet immediate. His nemesis (and really anyone’s) is the hedgehog. For Silver the fox-hedgehog distinction is simultaneously the most pertinent psychological fact about a person, the main cause of bad forecasting, and Dasein. The reader is given the impression that this is all settled fact.

For those who are not in the know, Isaiah Berlin discovered the fox-hedgehog distinction on an old Greek parchment. He then used it to solve the mystery of Schrodinger’s Tolstoy which stipulated that the Russian novelist existed in both ontological states at once. Because of Russell’s paradox, we know this to be impossible. According to Berlin’s solution, read first over BBC children’s radio, Tolstoy is in fact a hedgehog. Berlin is the fox.

After summarizing the concept’s history, Silver dismisses Berlin’s solution as silly because no serious thinker would force an artist into such a crude binary. Tolstoy’s being manifested according to a frequency best described by a binomial probability distribution. He most likely occupied both ontological positions simultaneously, albeit probably one more than the other.

Nevertheless, social scientists and public intellectuals do not deserve such nuanced treatment because according to Silver’s updated distribution, the likelihood that they’re a combination of both traits is zero (he goes out five decimal places). They're hedgehogs. Silver then describes the differences between foxes and hedgehogs as follows:

·         Foxes “[i]ncorporate ideas from different disciplines…regardless of their origin on the            political spectrum”. Hedgehogs “[m]ay view outsiders skeptically” (54).

·         Foxes adapt their thinking to changing facts. Hedgehogs change the facts to their pre-          existing biases.

·         Foxes are “self-critical”. They acknowledge their mistakes. Hedgehogs are “stubborn”.          They blame their mistakes on anything but themselves.

·         Foxes are “tolerant of complexity.” Hedgehogs “construct stories…that are neater and          tidier than the real world, with protagonists and villains, winners and losers, climaxes            and dénouements—and, usually, a happy ending for the home team” (58).

·         Foxes use statistics. Hedgehogs use their gut.

·         Foxes seek the truth. Hedgehogs want to be invited on Jay Leno.

This was news to me. I had never seen the fox/hedgehog distinction referenced in either my graduate level econometrics or game theory sequences. Nor was it ever invoked during research workshops. But this is easily explained if we were all hedgehogs. Perhaps social science departments should do a better job screening for us during the admissions process.

These were all, of course, sleepy thoughts. It was maybe 4:00 am now. My girlfriend sat up and said “we’re at Pando. We’re at Pando.” A fictional train station. She was sleep talking. I told her I’d be there in a moment. She said “ok” and rolled back over.

I wanted to go to sleep too, but Nate Silver’s criticisms of my former profession gnawed at me. I wasn’t quite sure what those numbers published in political science journals referred to anymore. I thought they were probability distributions. No, they most certainly were.

It occurred to me that Nate Silver might have a bit of a Clayton Bigsby complex. I looked at his Wikipedia page. My suspicion was quickly confirmed: Nate Silver’s a blind hedgehog who was raised by a family of foxes that treated him as one of their own.

Off the hook, I skimmed through a few more pages. It seemed that later on in the book he gives a bit more degreed hedging. But I didn’t let that contradict my biases.

It’s what Nate Silver would’ve wanted.