Diving Into Nate Silver’s Secret Sauce

From the beginning of this election season I have been highly skeptical of the odds Nate Silver has placed on various candidates winning various primaries.

by Mish

Silver’s odds, and his secret sauce reasons for them, are now in question in multiple places.

I sent an Op-Ed on this idea to the New York Times in January. As expected, they would not touch it. Since then, evidence against Silver has mounted in what’s clearly a “late to party” phenomenon.

Last weekend I had a phone discussion with Salil Mehta, a top-selling mathematics and statistics book author, about mathematical discrepancies and other biases in Silver’s estimates.

Mehta, investigated further and agreed with me in a report.

Mathematicians Welcome

Let me state upfront, for the second or third time, that I think Silver is brilliant. His calls on prior elections have been amazing.

However, his latest projections have been negatively amazing. I also question his math. Since he does not answer emails, at least mine, let’s start with a look at the math. Mathematicians welcome to chime in.

Here is a series of Silver’s “Polls Only Projections” at the time I captured them. Some results may have changed if you look at his Five-Thirty-Eight website today.

Wisconsin Polls

Note that the single most relevant poll had a weighting of 1.04. The only other positively weighted poll weighed in at 0.02.

I do not question Silver’s weightings. I presume he knows far more about which polls are more relevant and why than I do.

I do question his math, and his undisclosed secret sauce.

Via the magic of Silver’s secret sauce, Silver comes up with this projection. Once again, that projection has changed since my capture.

Wisconsin Weighted Averages

I looked at the preceding snapshot and said WTF?

Here are my calculations:

If we assume Silver is correct in his weights, the true weighted average of the poll weighted 1.04 and the second poll weighted 0.02, for Donald Trump is 34.9% not Silver’s 32.9%.

That is one of the things I spoke with Salil Mehta about. In addition, we discussed the odds of two 90+ percent failures in a short time span and other anomalies.

At first I assumed Silver missed reporting on a poll (now since out), but I found discrepancies in other states as well. Rather than post all of the Silver screen shots I captured I will just cut to the chase and show my tables.

New York

Based on true weighted odds, accepting Silver’s weights, I calculate Trump’s odds of winning New York at 60.7%. Silver says they are (were at the time) 58.8%


Silver’s Polls Only Projections, Not Polls Only

In every instance that I have investigated, Trump is on the losing end of these calculations.

Even Paul Krugman is commenting on “Tarnished Silver“.

In a discussion with Mehta regarding geometric weightings and other possibilities, neither of us could come up with an explanation other than what I label “secret sauce”.

In other words, Silver’s “Polls Only” projections are not polls only.

If you want another opinion, please consider Salil Mehta’s Mercurial Pollster Results.

New York Times Chimes In

Today the New York Times chimed in with How the G.O.P. Elite Lost Its Voters to Donald Trump.


I sent the New York Times an article on “attitudes” and Silver’s projections in January. Today, this concept is supposedly “news”.

The sorry thing about all this is one must give the Times 10 business days to respond.

Media Submission Recommendations

Here is my suggestion: Don’t bother with submissions unless you are some high profile person, in which case, no matter how inane your theory, the Times is likely to use your article.

To prove I am not biased against the New York Times, the same applies to the Financial Times, Bloomberg, and the Wall Street Journal, all of whom I have made timely submissions on various topics.

Your best hope of being published in mainstream media is

  1. Toe the line
  2. Have a big name (in which it does not matter how stupid a thing you write)
  3. Write an op-ed agreeing with something the media and big names agree with

The more timely your idea, the more you should consider sending it to me, ZeroHedge, or anyone else but mainstream media.

Mike “Mish” Shedlock