Gallup Blew Its Presidential Polls, but Why?

Last week's presidential election has widely been seen as a victory for pollsters who, on balance, saw President Obama as the favorite before Election Day. But that wasn't the case for the esteemed Gallup Organization. Its polling showed Republican Mitt Romney with a significant lead among likely voters 10 days before Nov. 6 and marginally ahead of Obama on the eve of an election that Obama won by about 3 percentage points.

At an event on Thursday at Gallup's downtown Washington offices, Gallup Editor in Chief Frank Newport told a gathering of fellow pollsters that the organization was reviewing its methodology in light of these inaccuracies. But its fairly consistent Republican bias in 2012 and its overestimation of the white portion of the electorate raise important questions about sampling and the way Gallup determines which respondents are registered and likely to vote.

"We don't have a definitive answer," Newport said.

The day before Election Day, Gallup released data culled from the four previous days, showing Romney with a 1-point lead among likely voters, 49 percent to 48 percent. Before that final survey, Gallup had suspended polling for three days in the aftermath of Hurricane Sandy, when nearly 10 million Americans were without electricity.

Immediately before the storm hit, Gallup showed Romney ahead by 5 points, 51 percent to 46 percent, and Romney led by as many as 7 points in mid-October. All the while, most other national polls showed a neck-and-neck race.

The reasons for Gallup's inaccurate results remain unclear, and they are particularly baffling because Gallup, from a simple methodological standpoint, does things the right way. The company calls both landline and cellular telephones, and in October, it increased the proportion of cell-phone interviews to half. Moreover, Gallup remains among the world's most prominent and respected public-opinion organizations, and its more than 75 years of polling data comprise a large portion of the information we have about Americans' attitudes about their government and society over that time.

So were Gallup's struggles this year the result of sampling bias ? through its random-digit-dialing interviews, did Gallup simply talk to too many Romney supporters? Was its likely-voter screen filtering out Obama supporters who would go on to cast ballots for the president? Or is it some combination of the two?

"Model" Behavior

Gallup's likely-voter model is a battery of seven questions it uses to determine which respondents are most likely to cast ballots. These questions include how likely they say they are to vote, their self-reported vote history, whether they know where to vote, and how much thought they have given to the election.

Respondents are awarded points for their answers to these questions, and only those who accrue a significant number of points pass through the likely-voter screen. But these measures may have led Gallup to understate the participation of the critical demographic groups that comprised Obama's winning coalition: younger voters and minorities.

On Oct. 26, Gallup released a demographic analysis of those respondents classified as likely voters in its daily tracking poll between Oct. 1 and Oct. 24. Of those voters, 78 percent were classified as non-Hispanic white, significantly more than the percentage of white voters measured by exit pollsters, 72 percent.

Four years ago, Gallup also found an electorate that was 78 percent white, an overestimation from the 74 or 75 percent recorded by exit polls. But this year's disparity is of a greater magnitude.

Gallup has also underrepresented younger respondents in its measures of likely voters. Over the first 24 days of October, 13 percent of Gallup's likely-voter sample was younger than age 30. Exit polls show these younger voters made up 19 percent of the national electorate.

Democratic pollster Mark Mellman believes this kind of likely-voter screen is counterproductive. Speaking at the event at the Gallup building, Mellman said that pollsters and the media have "overfetishized this whole notion of likely voters."

"We should not be concerned about finding likely voters," said Mellman. "We should be concerned about simulating the likely electorate." Implicit in Mellman's comments is the idea that the composition of that likely electorate would not have been 78 percent white.

Free "Samples"

But others say that issues with Gallup's initial sampling may be to blame. Andrew Kohut, the retiring director of the Pew Research Center who served as president of the Gallup Organization from 1979-1989, pointed to the difference in Gallup's results for all registered voters versus likely voters in its final preelection poll: Romney led by 1 point among likely voters, but Obama held a 3-point lead among all voters. Kohut compared that to Pew's final results, which also showed a discrepancy of 4 net points between registered and likely voters.

"I think there's something very basic going on with their sample. It's not with their likely-voter scale," Kohut told National Journal in a telephone interview, adding, "I helped create that likely-voter scale."

Emory University professor Alan Abramowitz wrote an article for the Huffington Post Pollster more than two weeks before Election Day titled, "Is Gallup Heading for Another Big Miss?" Abramowitz pointed to Gallup's overstatement of support for the generic Republican congressional candidate in 2010, compared to the overall number of votes cast for Republican congressional candidates in that election, as the basis for his assertion that Gallup is "the biggest outlier among all of the national polls."

"They need to rethink what they're doing," Abramowitz said Friday.

While Abramowitz pointed in his piece to the demographic composition of the likely-voter sample as a cause of Gallup's outlier status, he told National Journal, "The results for registered voters were also off."

Abramowitz referenced a long, exhaustive article written earlier this year by Huffington Post Pollster founder Mark Blumenthal that suggested the way Gallup asked its respondents about their race was causing them to weight their surveys to a population with more whites and fewer nonwhites. And while some of Gallup's methodological tweaks this year led them to interview more nonwhites, for the overall sample, "for some reason that didn't affect what they were finding" for registered or likely voters, Abramowitz said.

Overweighting

In the blog post announcing Gallup's switch to a 50 percent cell-phone sample, Newport also described the process by which Gallup weights, or adjusts, its sample. It makes adjustments to ensure that the initial sample of adults is reflective of the overall population?adjustments that Blumenthal wrote may be flawed?but it does not adjust further to simulate what Mellman, the Democratic pollster, calls the "likely electorate."

"As has always been the case, we do not attempt to weight the composition of the likely voter sample in any way?such as by political party or race or age?to approximate some guess of what we or others think it should look like demographically on Election Day," Newport wrote. "That approach is precarious given that the electorate can look quite different (especially looking at political parties) from one election to the next."

The problem for Gallup is that the electorate looked only slightly different from the most recent presidential election, and those differences reflected increasing roles for demographic groups more favorable to the incumbent, instead of the decreasing roles Gallup was finding.

After the election, Newport wrote in another blog post that he thinks "it is clear that voting today is subject to new pushes and pulls, including, in particular, the highly sophisticated ground games employed by the Obama (and, to a lesser degree, the Romney) campaign this year."

He continued: "These methods may in the end affect voters who were not certain about voting at the time of a poll interview, but who were brought into the voting pool at the last minute by aggressive get-out-the-vote and late registration methods. Our traditional 'bootstrap' method of identifying likely voters is self-weighting ? letting voters' responses to questions determine their probability of voting. This bears investigation."

But, as Pew's Kohut pointed out, his organization uses a very similar likely-voter screen, and their final poll was considerably more accurate. At the Thursday event, Newport conceded, "Everybody has a likely-voter model." Yet Gallup remained a Republican-leaning outlier over the final weeks of the campaign.

While the reasons for Gallup's GOP lean are not definitive, Kohut sounded wistful about the organization's poor performance. "I'm puzzled by it," he said. "I feel very badly about it. I wish they had done well."

Source: http://news.yahoo.com/gallup-blew-presidential-polls-why-080003934--politics.html

nbc news Heather Clem Con Edison LaGuardia Airport weather radar the weather channel tony romo