Election poll accuracy in doubt with disparity, outdated methodology

At this time four years ago when President Obama was running for re-election, there were no polling outliers.

Run down the list of 15 top pollsters, and their results in late October 2012 on Real Clear Politics. No matter the firm, the national numbers were within a few points of one another. Mr. Obama was up 1 or 2 points, or Mitt Romney was tied or up a point.

What a difference four years, and perhaps Republican Donald Trump, make. National polls are anything but uniform. Perhaps this disparity was signaled in 2014, when pollsters badly missed in a number of key races by undercounting Republican voters.

Four respected firms — Gravis Marketing, Rasmussen, USC Dornsife/LA Times and IBD/TIPP — for weeks have put the contest either at a statistical dead heat or up or down a point or two.

Then The Associated Press came out with a wow poll Thursday showing Mrs. Clinton up by a whopping 14 points.

“Hillary Clinton appears on the cusp of a potentially commanding victory over Donald Trump, fueled by solid Democratic turnout in early voting, massive operational advantages and increasing enthusiasm among her supporters,” AP said.

Pollsters have taken note of the large disparity this time.

“This is an unusual election in a lot of ways,” said Peter A. Brown, assistant director at Quinnipiac University Polling Institute. “Both candidates are disliked by more than half of the voters. In this kind of environment, anything seems potentially possible. That doesn’t even count the difference in methodology from other polls.”

Then there is the question of reaching voters. It’s harder, as landlines disappear and cellphones take a larger share. Robo-pollsters are banned from calling smartphones. Live survey pollsters must judge the right mix of landlines and cellphones.

“The whole polling industry is in flux,” said a statement from the robo-calling Rasmussen Reports.

“With the death of the landline, we are all scrambling to find alternative survey methods that give us valid and demographically accurate samples,” the polling firm said. “We now use anonymous online panels like many pollsters do to reach voters under 40 and minorities in particular.”

Polling is part science, part art. All start with computer-selected, randomly called telephone numbers (unless it is an internet poll). Respondents are questioned by a person or a machine.

Then comes the hard part: boiling down the sample into a turnout model that contains the projected number of blacks, whites, Hispanics, men, women, Democrats and Republicans. That is the “methodology.” Some polls predict Mrs. Clinton will garner a larger percentage of blacks compared with President Obama. Others factor in smaller percentages.

“We do know some pollsters are putting a lot more independents in their model this year, compared to past years,” Rasmussen said. “Also, their turnout projections can make a world of difference. If, for example, you project that Democratic and Republican turnout will be comparable to 2012, you’ll have polls that favor the Democrats because Dems overperformed and GOPers underperformed four years ago. Blacks overvoted; whites undervoted. But is that really valid? Will Hillary Clinton generate the kind of turnout Barack Obama did? That seems doubtful.”

One feature in some polls tends to enrage conservatives. They see a larger sample of Democrats in a poll’s “internals” than Republicans and think a poll is “rigged,” to use a Trump word.

Pollsters say that in most cases they simply ask, “With what political party do you associate or identify?” The answer typically follows which candidate does best in that particular poll. It is not, in other words, a conscious effort to include more Democrats than Republicans, the pollsters say.

An illustration of how volatile the race is: An ABC/Washington Post tracking poll had Mrs. Clinton up 12 points. A few days later, it shrank to 4 points. On Sunday, it was 1 point. ABC and its partner had joined the outliers.

The Associated Press’ 14-point projection would thrust Mrs. Clinton into landslide territory. She would carry a string of red states and quash Mr. Trump’s hopes of carrying Ohio and Florida.

That scenario creates skepticism about a 14-point lead, given that Mr. Trump leads in some polls in those two states.

The AP poll is not close to other polls. But there are polls that diverge from the “dead heat” surveys. Fox News and Reuters each has Mrs. Clinton leading by 4 points. CNBC has her up 9 points.

The bottom line: There is no national uniformity, except that no poll has Mr. Trump with a significant lead.

The polling chaos has reached into individual states and candidates. Two TV station polls conducted at about the same time in New Hampshire: Republican Senate incumbent Kelly Ayotte is down 9 points in one and up 4 points in the other.

In Pennsylvania, two polls — Emerson College and Quinnipiac — show Republican Senate incumbent Patrick J. Toomey up 3 points and 4 points, respectively. But The New York Times says Democratic challenger Katie McGinty is up 3 points.

In Wisconsin, Republican Senate incumbent Ron Johnson is only slightly behind Democratic challenger Russ Feingold, according to polls by Marquette Law School, which has an excellent track record in the state, and by CBS News. But a poll from Wisconsin’s public radio station shows Mr. Johnson hopelessly behind by 12 points.

“This has been a nontraditional election, and that has played out with a more volatile electorate,” said Spencer Kimball, a professor and adviser to the Emerson College Polling Society in Boston.

Bloomberg News judged Emerson one of the most accurate pollsters during the primary season.

“We have seen in over 40 state polls that we have conducted a variance in the results when compared with other polls taken around the same time,” Mr. Kimball said. “For example, in Nevada and New Hampshire, Senate polls have varied by 8 to 10 points with different projections of the winner.”

The U.S. Senate race in Nevada between Democrat Cortez Masto and Republican incumbent Joe Heck is truly wild, pollingwise. An Oct. 24 NBC/WSJ/Marist poll has Mr. Heck up 7 points. The next day, Gravis Matketing said Mr. Heck was trailing by 6 points — two separate polls with a 13-point difference.

One reason for the volatility, Mr. Kimball said, is that voters keep switching, such as supporters of Ohio Gov. John Kasich, a Republican.

Still, he said, “as the Election Day nears, we typically see partisan voters return to the base and expect to see that reflected in the poll numbers next week.”

The off-year results of 2014

Is it possible that polling has reached an age of uncertainty? That internet devices, diminishing landlines and a larger force of independent voters have turned a science into an educated guess? Did the 2014 off-year election signal this new era?

The question is pertinent because major pollsters missed some significant races in 2014 — by a lot. In each case, the Republican vote was undercounted.

The Washington Times examined the final results in key contested races that year:

• Maryland Gov. Larry Hogan, a Republican.

Major pollsters had Mr. Hogan losing by double digits in dark-blue Maryland. The New York Times/YouGov poll had him down 13 points. The Washington Post said he was 11 down.

Mr. Hogan won by 3.8 percentage points.

• Sen. Mark R. Warner, Virginia Democrat.

Mr. Warner was an odds-on favorite to win big in Virginia, according to major pollsters. The New York Times/YouGov poll had him up 10 points. The oft-quoted Public Policy Polling: 9-point lead. A local university predicted he would win by 20 points.

Mr. Warner beat Republican challenger Ed Gillespie by the thinnest of margins: 0.8 percentage points. The major polls were not close.

• Sen. Pat Roberts, Kansas Republican.

Most major polls right up to Election Day showed Mr. Roberts tied with independent (Democrat-leaning) Greg Orman. Four polls had Mr. Orman ahead.

Mr. Roberts won by 10.6 percentage points.

• Sen. Joni Ernst, Iowa Republican.

The major pollsters had this race far closer than it ended. YouGov had Democrat Bruce Braley winning. Quinnipiac had it a 47-47 tie. If any major poll had Ms. Ernst winning, it was by a few points.

Ms. Ernst won by 8.3 points.

(Quinnipiac now reports that Mrs. Clinton and Mr. Trump are tied in Iowa.)

• Kentucky Gov. Matt Bevin, a Republican.

This was one of the few major 2015 races, but it followed a pattern of respected pollsters undercounting Republican turnout. All major pollsters had Mr. Bevin losing to Democrat Jack Conway. The Survey USA, for example, said he was down 5 points. Mason Dixon said 2 points.

Mr. Bevin won by 8.7 points.

Meanwhile, pollsters correctly predicted that Mitch McConnell, Kentucky Republican, would win re-election to the U.S. Senate in 2014 but misjudged his margin. Most had him at 6 to 10 points. He won by 15.5 points.

The big pollsters did not miss every key race. In Colorado, for example, most had Republican Sen. Cory Gardner edging Democrat Mark Udall.

There is the possibility that 2014 was a “Republican year,” as they say, with quirky turnouts. Turnout this time will be much larger, perhaps bringing out a larger share of Democrats. If anything, 2012 polling by major firms underestimated Mr. Obama’s final margin of victory by a few points as opposed to underestimating Republican Mitt Romney’s tally.

As for today’s polls, Kellyanne Conway, Mr. Trump’s campaign manager, said on “Fox News Sunday”: “Some of these polls, if you’re only using listed sample and you’re identifying people who voted Republican in 2012, or voted Democrat in 2012, and you’re not really bringing in people who haven’t voted in a long time, the lapse voters or first-time voters who are truly enthusiastic about Donald Trump as an outsider trying to really shake up the system.”