One of my duties as an American immigrant in Taiwan is to answer questions from local Taiwanese about what is really going on in America these days. And, although I’ve lived here pretty much continuously for over the last two years, I do pay attention to what is going on “back home,” so I’m happy to offer my opinion.
The question I’m asked most often recently is, of course, who will win the US Presidential Election. Sometimes this question is asked with some level of incredulity, as in “do I really think Biden is going to beat Trump?!” or “how can Trump still be competitive, given issues X, Y, Z, or W?!” Another common packaging, one that usually leads to an interesting discussion, is the subject of today’s column. Namely, “I’ve seen a few polls that have Biden leading, but WHAT IF THE POLLS ARE WRONG?!”
The polls *are* wrong. That is, polling by its nature is an imperfect approximation of the views of the electorate, and in fact almost every decent poll and pollster explicitly acknowledges that they are wrong, and attempts to quantify how wrong they may be in the form of a “margin of error.”
Beyond “margin of error,” different polling organizations, polling the same race on the same days, regularly come up with different answers, sometimes even outside of each other’s margin of error. For example, ABC/Washington Post might find Biden leading among likely voters nationally by 12 points, while another organization might have him leading by only 8, or as much 16, or even as little as 4.
Faced with this disparity, people will ask, “which poll is right?” or “which poll is best?” and many will end up latching onto a particular poll that confirms what they WANT TO BE TRUE, and defending that poll as “right,” and all the other polls as “biased” or “wrong.”
Polls differ in methodology, particularly in how they represent the electorate, reach voters, and decide who is more likely to vote versus not. Certain well-respected polls tend to favor Republicans, and other equally well-respected polls tend to favor Democrats, but these tendencies are usually the result of methodology, rather than a result of intentional manipulation or bias.
Enter sites like FiveThirtyEight, and others like it, that construct “forecast models,” not based on the data in any one poll, but instead on aggregated data across many polls, along with other factors such as the economy, the effect of incumbency, the amount of attention being paid to the race, and, importantly, the historical quality of the polling organization, as measured by correlations between many thousands of different poll results and the actual results of previous elections.
For example, instead of deciding that the ABC/Washington Post poll is “right” (because you like the answer), models like FiveThirtyEight instead assume that all polls, including polls that may be your favorite, are WRONG, and attempt to model HOW WRONG they might be by looking at many other polls, both at the national level and the state level.
In this recent example, Reuters/Ipsos also conducted a poll on the same dates, and also found that Biden was leading by 12. USC Dornsife conducted polls over two different likely voter panels from Sep 26 - Oct 9 that gave Biden an 11- and 12-point lead, respectively.
So, rather than ask who is right and who is wrong, FiveThirtyEight takes all those results merely as pieces of imperfect evidence, and weighs them against many other pieces of imperfect evidence to create a new estimate of the election result.
In this case, the preponderance of the evidence, according to FiveThirtyEight, is that a 12-point lead is probably an over-estimation of Biden’s support, and as of 10/11, they have Biden’s lead at “only” 10.1 points (still a large lead, and the largest Biden has enjoyed since the model went live in March of 2020). Just as recently as a few days ago, the lead was forecast to be only 9 points, even though we had seen polling at that time that had Biden up by as much as 16.
I rely almost exclusively on forecast models when I offer my opinion about what I think will happen in the election. I first encountered FiveThirtyEight in 2008, around the time that Nate Silver revealed that he was the author of its model and the creator of the (rather small) website that hosted the predictions. Nate is a data scientist and journalist by training, and I first encountered his work not in the context of politics, but rather in the context of a player forecasting model for a popular baseball statistics website.
Since FiveThirtyEight’s launch, two things I have appreciated about it (as kind of a math geek myself) is that Nate is 100 percent transparent about how the model works, and, importantly, he doesn’t change the model “on the fly” in order to respond to events or predictions that look odd, or, at times, maybe flat out wrong. He builds the model, makes his assumptions clear, and just keeps it running it day after day as new polling comes in.
Nate’s model nailed the 2008 election with eerie precision, and it repeated the feat again in 2012. Quite simply, his work revolutionized and popularized election forecasting, and he became something of a hero to a generation of nerds and math geeks, who have appreciated seeing work from their field applied to politics in such a compelling way.
These days, Nate takes TONS of flak for his models and his predictions. He’s accused of being biased towards liberals (to provide propaganda to ensure that liberal candidates are elected), or biased towards conservatives (to keep races “artificially close,” thus ensuring traffic to his website); he’s subject to all sorts of amateur critiques of his methodology and his assumptions -- everything from how he treats undecided voters, to early voting, all the way to esoteric mathematical details about how he calculates “house effects” or some other component of his model -- some of which are valid and worthy of discussion, but many of which are a fancy way of saying, “we don’t like your results.”
More than anything else, he continues to take an ENORMOUS amount of heat for being famously and historically “wrong” about the 2016 election, in which Trump pulled off the most unexpected election upset since Truman defeated Dewey in 1948. What’s interesting to me about this criticism of FiveThirtyEight in the aftermath of 2016 is that Nate’s model actually gave Trump a better chance than almost any other (legitimate) model produced at the time. In 2016, FiveThirtyEight gave Trump almost a 30 percent chance of winning, in contrast to many who had Trump’s chances at below 10 percent, and sometimes even below 1 percent. On November 8, 2 days before the election, FiveThirtyEight had Trump at a 35 percent chance of winning.
Furthermore, the conclusion that people draw from 2016, that Trump’s election PROVES that the “polls are wrong” is something that Nate would 100 percent agree with. The fact that polls can be wrong is indeed why FiveThirtyEight exists in the first place.
The big question then is what went wrong with FiveThirtyEight in 2016, and can we reasonably expect the same thing to happen again in 2020? I apologize in advance for the detailed forensics that I am about to subject you to, but I believe that some of these small details are necessary and useful to understanding the situation in 2020.
On the morning of Election Day 2016, FiveThirtyEight’s final prediction for the Presidential election was displayed on their website as follows.
Clinton was forecast to win the popular vote by an estimated 3.6 points nationally, and she was predicted to win the Electoral College* by an average of 302 - 235. This forecast is probabilistic, and, behind the scenes, the model was predicting a broad range of outcomes for the election (from a Trump win to a Clinton blowout), along with the probability of each outcome occurring. The map above is kind of an aggregate of all those predictions.
Now, to understand what went wrong in 2016, we need to focus on five states -- FL, NC, PA, MI and WI. These states should look familiar to folks following the 2020 election. Pollsters, and the campaigns themselves, are obsessed with these five states in 2020, and what happened in 2016 is the reason why.
Clinton was favored to win all of these states. She was a very slight favorite in FL and NC (as indicated by the light blue), but according to the probabilities on FiveThirtyEight, the outcome in both of those states were essentially a coin flip (Trump had an almost 50 percent chance of winning each state). What’s more, the outcomes in those states are NOT INDEPENDENT OF EACH OTHER. Both states share similar overlapping demographics, and so a polling “error” in FL would probably manifest as a polling error in NC, as well.
One key feature of the FiveThirtyEight model is the recognition of this correlation between states. The US election is not 50 independent state elections, but rather 50 elections whose results are correlated with each other to some degree, and to the results across the nation as a whole. What this means in terms of the model’s performance is that, normally, if you had two states that were both 50/50, the chance that a candidate would win both of them is only 25 percent (½ * ½ = ¼). However, because the results are correlated, the chances of both states voting the same way is much higher.
This correlation also applies to the next set of states, PA, MI and WI, where Clinton was favored more heavily. The model here gave Trump “only” a 20 percent chance of winning each state, but because the performance of these states is correlated, the chance of winning all three is not 8 percent, but instead much closer to 20 percent. Trump was surely an underdog, but his defeat was by no means a sure thing.
What happened, of course, is that Clinton lost all five of these states, and Trump won the election. FL and NC broke for Trump, and, more significantly, the model underestimated Trump’s support in PA and the Upper Midwest, so he was able to narrowly win those states, too, with a margin of less than 1 percent. Clinton did win the popular vote, but by only 2 points instead of the predicted 3.6, and that level of Clinton underperformance was pretty consistent across all the states, even in the ones that she won.
So what should we take away from all this? First, yes, the model was “wrong” and the model (along with the polls that fed into it) overestimated Clinton’s support. But it wasn’t WAY OFF. There were enough close races such that this relatively small over-estimation in support -- a 1.5 percent miss nationally -- led to a big change in the Electoral College.
The second takeaway is that events with roughly 25-30 percent probability, while not the most likely, are not all that UNLIKELY, either. It’s only 25 percent likely that someone would draw a Heart from a 4-suited deck of cards, but almost no one would be surprised if it happened. Sometimes you just get unlucky. And, in the case of elections, sometimes you get a little unlucky, but are also missing information that would have helped you make a better prediction.
One piece of information missing in 2016 was how undecided voters were going to behave on Election Day, especially in light of a late-breaking “scandal” that was re-opened on October 28, near the end of the campaign. In 2016, there were an unusually large amount of undecided voters until very late in the Election cycle, and, as it turned out, a very large percentage of these late deciders voted for Trump.
A second crucial piece of information that was missing in 2016 was polling data from exactly the Upper Midwest states that were the key to Trump’s upset. Obama had won these states easily in 2008 and 2012, and many assumed that Obama’s “Blue Wall” in the Upper Midwest would hold. Infamously, Clinton didn’t even really campaign there in the late days of the race, choosing instead to focus on states like FL, NC and Nevada, where the polling was closer.
So how similar is the situation in 2016 to our current situation in 2020? I think there are at least three big differences between now and then.
1. Biden’s forecast lead in the national popular vote (10.1 percent) is much greater than Clinton’s (3.6 percent). Clinton’s support was overestimated by about 1.5 percent nationally. If pollsters would miss by that same amount this year, Biden would still lead by about 8-9 points nationally, and, importantly, it is nearly impossible, even with Trump’s advantage in the Electoral College, to win the national popular vote by 8 points and still lose the election. If Biden wins the popular vote by 8.5, he will win the Electoral College comfortably.
2. There are far fewer undecided voters in 2020 than in 2016. Neither Clinton nor Trump, at no time in 2016, polled above 50 percent, and there was a relatively large and lingering set of undecided voters (and voters who were considering 3rd party candidates) late into October, on the order of about 10-12 percent of the electorate. In 2020, Biden polls regularly above 50 percent, and undecided voters, even in the most contested states, number on average only about 4-5 percent.
3. There is a lot more polling this year. Perhaps traumatized by the upset in 2016, pollsters have been much more active this cycle, not only in battleground states like MI, WI and PA, but across the rest of the country, as well. Some relative blind spots do exist but any state that has been traditionally competitive, with the possible exception of Nevada, has been extensively polled. Even states like Texas, which has been very solidly Republican for quite some time, are seeing their fair share of polls. More polls mean that errors in any one poll tend to get cancelled out (it’s just one piece of data among many), and the data we get from polls is fresher and more up-to-date.
It is for these reasons that, on November 3, I give the edge to Biden. As of today, FiveThirtyEight gives Trump a 15 percent chance of turning things around and winning, and, as we discussed, events with a 15-percent likelihood are merely unlikely, and not impossible. No one should be shocked if some Black Swan event occurs, and Trump cobbles together enough electoral votes to win.
For that to happen, however, something big will have to change between now and November 3, something that causes a large disruption of the race and causes voters to change their minds. 2020 has been a crazy year, and there remains a large amount of uncertainty and doubt about the election, but there are simply not enough undecided voters for Trump to make up the gap with Biden, even if the polls are wrong.
Trump needs a Big Thing to win, and although he may still get it, I don’t believe that the Big Thing could be as simple as a polling error. Forecast models give us a good way to think about the inherent inaccuracy of polls, and, at this moment, what they are telling us is that Biden will win in November.
*Electoral College
Unlike in Taiwan, the winner of the US Presidential Election is not determined by who gets the most votes nationally on Election Day. Instead, America relies on a body of electors called the Electoral College, whose sole purpose is to come together every 4 years to vote for the US President. This “second election” takes place in early December, although it is largely considered a formality.
Each of the 50 states in the US is allocated a certain number of electors in the college, based on that state’s representation in the US Congress and thus indirectly on their population. There are 538 electoral votes available in total, and therefore 270 votes are needed to win a majority.
On November 3, each state will hold a statewide election in order to determine who to designate as electors. Voters don’t really know who the electors are; instead, they vote for their presidential candidate of choice, and ,ost states adopt an “all or nothing” approach to designating electors -- the winner of the popular vote in that state gets a full set of electors who have pledged to vote for him or her.
The reason that states like FL get a lot of attention in US elections has to do with the Electoral College. First, it is a populous state, and therefore appoints a lot of electors (29, or more than 10 percent of the electors needed to secure a majority). Second, elections in FL are often competitive, so those 29 electors are considered “up for grabs.” So, while there are states with more electors (e.g, CA designates 55, but hasn’t voted for a Republican in over 30 years), Florida, and other medium-sized states like PA, offer more “bang for the buck” in the race for a majority, and thus get more attention.
**For the Chinese version please check: 美國總統大選：萬一民調（又）錯了怎麼辦？
執行編輯：邱佑寧
核稿編輯：林欣蘋
