Pollsters say they are listening — even about their accuracy

Pollsters don’t need to conduct a poll to see the general public’s opinion of them. Most, if not all, are well aware that they’re under immense scrutiny when predicting this year’s midterm election outcomes after the 2016 and 2020 presidential elections brought an even harsher spotlight onto their predictions.

Outcry over polling inaccuracies is certainly not new, whether from the news media’s talking heads or the average voter. Pollsters do listen, and with each election cycle come updates to a firm’s methodologies. But certain aspects of polling are set in stone and cannot be changed.

One example is calculating the margin of error, a standard feature to account for possible miscalculations. The seemingly arbitrary number is actually a mathematical equation, an axiom that doesn’t change — ever.

This begs the question: Well, if it’s based on something as solid as an equation, why do polls seemingly grow more and more inaccurate? Why did 2020 polls feature polling errors of unusual magnitudes even after widespread criticism of 2016 polling standards?

Eric Tollar, deputy director of Emerson College Polling, said the biggest problem he’s seen in his industry is his colleagues’ unwillingness to adapt to changing technology.

“For years, the gold standard of data collection was in-person interviews, and there are a lot of organizations that continue to use them,” said Tollar. “Well, in-person interviews are terribly expensive. And then hiring a phone bank to do all of that can be difficult.”

In order to adapt to a changing demographic and make polls both accessible and understandable to young voters, Tollar said updates like interactive polling reports, videos, online panels, live operators, texting, and Integrated Voice Response are all necessary for accurate polls.

“Getting voters the age of 18 to 24 is miraculously hard; they just don’t answer the phone when strange numbers call,” he said. “Different modes of data collection have different strengths within different demographics, and being acutely aware of which works for what is really the key to success, as this market continues to evolve.”

Tollar also stressed the importance of transparency in the polling universe and encouraged others to release complete datasets.

“A lot of times, pollsters will just put out their top lines and they won’t tell you their turnout model,” he said. “How did you come to that conclusion? When you release the full cross tabs and full tables, you get to see all of that, and I think that’s key.”

Having that information readily available isn’t just beneficial for gaining the public’s trust, it also allows pollsters to do more constructive post-mortem analyses.

“Any responsible pollster will go back into their data and figure out what bombs so you don’t make those mistakes again,” said Tollar. “Pre-election polling is an extraordinarily valuable way to see if you’re right or wrong. It’s a great way for you to go back and check your work.”

David Paleologos, director of the Suffolk University Political Research Center, said another aspect of polling that has changed is people and their attitudes towards polling in such a politicized state.

“It’s vicious, vicious, vicious,” said Paleologos. “[The amount of] people that hang up and swear at us when we call, how rude people are to the callers — this is all new. And of course, there are the people that just don’t trust anything. They don’t trust the electoral process. They don’t trust politicians. And they certainly don’t trust pollsters.”

Public distrust of polling, especially in the last six years, has risen to new highs, especially with Republicans who believe polls are biased against them. Others point out that difficulties in accessing voters may skew a poll in one direction or another.

“What a lot of people don’t understand is that just because it’s more difficult to reach a young person or a particular race or a particular demographic, doesn’t mean the poll is wrong,” said Paleologos. “It just means we have to make that many more attempts to get somebody that represents that characteristic that represents that group of voters, so it may take us 20 tries instead of 16.”

Paleologos added that one poll outcome can be much more appealing than another to cynical audiences looking for inaccuracies, even if pollsters are right most of the time.

“It’s sexy to say a pollster is way off or wrong,” he said. “It’s like being a weatherman. The weatherman never gets credit for the weather being right the next day, but the one time they miss a snowstorm or something, they get humiliated. And that’s what happens to us.”

For 2024 and onwards, Paleologos said he wants the public to be more educated about the advantages and limitations of polling so that it can be used as a tool, not a weapon.

“It should be used as something that can help explain phenomenon rather than be used as a fundraising mechanism for a candidate,” he said. “And it’s important for people to not fixate on the few polls that miss by a longshot.

“What we put out isn’t meant to be exact. It’s meant to be within a range of points. But people interpret it as like, ‘Oh, he said that they were gonna win by nine points but they only won by three or four,’ which means, well, the poll was right. It’s probably within the margin of error.”

Ultimately, without acclimating to a changing environment, pollsters’ efforts are doomed to fail, threatening a powerful force that helps guide policy. Polling, done correctly, is still the best way to hear citizens’ opinions. Elections show who won, but polling shows why.

“If polling is dead, show me what’s next, and I’ll be the first person to tell you that you’re right,” said Tollar. “Because for me, it’s about doing it the right way. So if there’s a better way for political campaigns to run, show it. Until then, we just have to work on using technology to make polling better.”

Kana Ruhalter writes for the Athol Daily News from the Boston University Statehouse Program.