×
×
homepage logo
LOGIN
SUBSCRIBE

Polling the polls

By ERICK ERICKSON - | Feb 22, 2020

The polls are not wrong. I know many people think polls are wrong, because the polls say things people do not believe. Sure, there are bad pollsters, but polling averages do a pretty good job. Take 2016, the example that comes to mind for a lot of people who say polls are wrong.

Hillary Clinton did, in fact, win the popular vote. The final polling average had her winning by 3%. She won by 2%. The polls were right. The problem is that we elect the President with an Electoral College, not a popular vote. The individual polls of Wisconsin, Michigan and Pennsylvania were far more instructive. But individual state polling is hard to do, and it’s more expensive.

As a general rule, if a poll has the candidates with a gap of 5% or more, those numbers tend to trickle down proportionally at the state level. When polls show candidates with margins of less than 5%, go to individual state polling to get a more accurate reading of a presidential race.

The methodology matters, too. Let me give you the methodology of the Emerson College poll. According to their latest national poll, which has Sen. Bernie Sanders with a 7-point lead, “Data was collected using both an Interactive Voice Response (IVR) system of landlines only … and online panels provided by (Amazon) Turk.” In other words, the poll only called landlines, not cellphones. It used a robocall, not a live operator. It also used an online voting system that involved Amazon Mechanical Turk. Amazon describes its Turk system as “a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually.”

That is not a good method of polling. There were no live operators and no cellphones. Compare Emerson to the ABC News-Washington Post poll, which describes its methodology as, “conducted by landline and cellular telephone Feb. 14-17, 2020, in English and Spanish, among a random national sample of 1,066 adults.” That poll has Sanders with a 15-point national lead.

The ABC News-Washington Post poll uses a good methodology. They used landlines and cellphones. They polled randomly. They also did it in Spanish. That’s pretty comprehensive.

Pew Research has out a report on bogus respondents in online polling. They found “that online polls conducted with widely-used opt-in sources contain small but measurable shares of bogus respondents (about 4% to 7%, depending on the source). Critically, these bogus respondents are not just answering at random, but rather they tend to select positive answer choices — introducing a small, systematic bias into estimates like presidential approval.”

In other words, beware of pollsters using online groups to participate in polls.

Polling really is a combination of art and science. There are great pollsters out there. The media overly relies on polling to help shape news coverage, but it can be helpful. We can, for example, see the rise of Michael Bloomberg in the polling. He has not yet been on the ballot, but we can tell his spending on ads is helping him. This, in turn, tells us that major ad-buying actually does help by boosting name identification.

But Sanders is now leading in the polls. Joe Biden has what can best be described as electile dysfunction. Biden led the polling nationally for more than 52 weeks. Before Biden even formally declared, he held about 28% in the polling averages. But his numbers are starting to plummet as Sanders surges and Bloomberg takes off.

It is an open question if Bloomberg can compensate for the fact that he has not been on the ballot yet and now has had a terrible debate performance. Super Tuesday is on March 3, and Bloomberg will finally be on a ballot. In the meantime, don’t dismiss the polls. Just check the methodology. Also, for all the horror stories about President Donald Trump losing to everyone in the national polls, go check out the swing states. In Michigan, Pennsylvania, Wisconsin, Iowa, Florida, North Carolina and beyond, Trump actually does far better than national polls suggest. That could make for an interesting election.

Newsletter

Join thousands already receiving our daily newsletter.

Interests
Are you a paying subscriber to the newspaper? *