A model that missed too many likely voters, undercounted Obama supporters on the coasts and relied too heavily on the telephone book helped produce Gallup’s very public miscall of the 2012 presidential election, the venerable polling organization acknowledged in an internally produced report released to the public Tuesday.
Company officials said the firm was determined to do better in the wake of the 2012 debacle. While most pollsters declared President Obama the Election Day favorite, Gallup’s final pre-election estimate of the popular vote showed GOP nominee Mitt Romney with a narrow 49 percent to 48 percent lead over Mr. Obama, with a margin of error of 2 percentage points. In the end, Mr. Obama won 51 percent to 47 percent for Mr. Romney.
All pollsters take the raw data from respondents and adjust the numbers based on their estimates of who will actually turn up at the polls on Election Day. But Gallup concluded that, compared to most of its polling rivals, its adjustment model was skewed too much in Mr. Romney’s favor.
“Gallup’s four-point shift toward Romney, however, was greater than other polls’ shifts,” the report said. “And various combinations of the use or weighting of ’likely voter’ questions would have changed Gallup’s final estimate of the presidential vote to be more in line with other firms’.”
Gallup’s surveys were a point of continuing controversy during the 2012 vote, with the Obama campaign and many private polling analysts questioning the company’s take on the race and its estimate of the likely composition of the 2012 electorate. Gallup chief pollster Frank Newport said Tuesday the firm will continue to investigate its methods and consider changes, centered on a number of areas identified by the post-vote studies, including:
• Likely voter estimates — The report found that the poll’s estimate of likely voters, which involved the use of seven questions to score a respondent’s propensity to vote, placed too much emphasis on past voting behavior and on asking voters about “thought given to the election.” While most polls understated Mr. Obama’s final victory margin, Gallup was among the least accurate.
• Geographic distribution — In part because of time-zone bias, Gallup conducted a disproportionate number of interviews in the central part of the country, while undercounting East and West Coast areas, including some strong pockets of Obama support.
• Race and ethnicity — In part because of the way its questions on race and ethnicity were structured, Gallup’s polls produced “a disproportionate number of respondents reporting they were multiracial and American Indian/Alaska Native,” the report found.
Gallup has since updated how it collects race and ethnicity data.
• “Listed landline bias” — Gallup conducted half of its telephone surveys via cellphone and half via landlines, but the pollster concluded that using listed landlines instead of dialing random home phone numbers produced “older and more Republican” respondents.
Dialing random landlines, instead of relying on listed landlines, would have reached a demographic that was younger, more Democratic and more likely to lean toward Mr. Obama. Because of these differences, Gallup will return to using randomly-dialed, “list-assisted” numbers for their landline surveys, a method used prior to 2011.
Please read our comment policy before commenting.