Op-Ed: How I Analyze Polls
November 3, 2020
Polls are an important way to follow elections, and politicians and journalists alike rely on these numbers to see how well a campaign is doing. However, polls are not meant to predict the outcome of elections—they are only used to demonstrate which way a state is leaning.
So, how are these numbers analyzed?
It is important to look at data from different sources, like RealClearPolitics and FiveThirtyEight. When it comes to analyzing polls, there are a few key terms to keep in mind: the characteristics of a state, the pollster grade, how many people were polled, and the margin of error.
There are several characteristics that a state can fall under. These terms are safe (or solid), likely, lean, tilt, and effective toss-up states. The classification of each state is is determined by examining what percent of those polled leaned in favor of each party.
The chance of victory (85% chance) or percentage of the popular vote won, (48-46, 2% victory) defines each of these categories. ARXIV says that a safe state has a greater than 95% chance of victory for one party, 75%-95% is likely, 60%-75% is tilt, and less than 60% is a toss-up.
For percentage of victory, everyone uses a different model. I believe that less than 1% victory is a tilt, 1-5% is a lean, 5-15% is likely, and greater than 15% is safe. States like Massachusetts (36.9%) and California (29.5%) are safe Democratic States with each polling at around a 30 percent electoral victory.
Not all states are solid states. In fact, many elections have states that will “swing” in favor of a candidate. That is why some states are called swing states—or battleground states. These states are not expected to consistently vote in favor of one party, especially Florida, North Carolina, Ohio, and New Hampshire. This year, however, there are more swing states than usual, with Georgia, Arizona, and Minnesota also in play. Battleground states tend to fall in the lean, tilt, and tossup categories, and they are usually decided by less than two percent.
FiveThirtyEight collects data from numerous polls, but keep in mind that not all polls are equal. FiveThirtyEight rates each pollster to give a better sense of which polls are most accurate. They base these rating off their “predictive +/-.” The predictive +/- tells how reliable the polls are and how accurate they will be in the future. The best polls have a negative predicative value. The grading system goes from A+, the best, to F, the worst.
Polls should not always be used to predict the outcomes of elections because the number of people polled varies. Every time you see a poll, there are only a select number of people polled. Let’s use North Carolina in this poll as an example. You will find that next to the name of the pollster there is a sample size and two letters to indicate the type of voters. There are four different types of voters that pollsters look at, which include LV (Likely Voters), RV (Registered Voters), V (Voters), and A (Adults). For example, the bottom row indicates there were 627 likely voters polled, clearly not enough to indicate the outcome of the election.
Lastly, it is imperative to consider the margin of error in polls, which RealClearPolitics includes in its margin of error data.
If we look at the same poll as before, you will see the margin of error (MoE) is 4.9. Since the outcome of the poll was a tie, this means that Biden could win by 4.9 points and Trump could win by 4.9 points. So, to say that a poll is wrong means it must be outside the margin of error.
Another example would be an Ipsos/Reuters poll that says that Biden is ahead by 1 point with a margin of error of 4. If Trump happens to beat Biden in North Carolina, the poll is not automatically incorrect. If Trump wins by 2 points, this poll is still accurate. However, if Trump wins by 5 points, then this poll would be deemed incorrect.
If nothing else, remember that polls vary in their accuracy. Even though there are accusations about polls being falsified or rigged, this is not generally the case. Even in the 2016 presidential election, the polls were mostly on point, thanks to the margin of error. Just remember that polls aren’t the ultimate deciding factor, they are merely indicators of which way states are trending.
Mark Hytha • Nov 4, 2020 at 10:11 am
Another great article Evan, and a very good time to be writing about this important topic. Keep up the good work!