Sunday, October 05, 2008
Many variables in political polls
Related
The latest from our Blue Ridge Caucus politics blog
- Gilbert expresses regret for abortion remarks
- Senate rebuffs McDonnell, rejects teacher contract change
From The Roanoke Times
From the Newsroom column
Dwayne Yancey, Senior editor for new channels
981-3113
From the Newsroom blog
Recent posts
The polls this election season are much like the weather in certain cities: If you don't like the way things are now, just wait and they'll change.
Take Virginia, for instance, which hasn't been this fought over since Lee and Grant had their last go-round in 1865. In the past few weeks, we've seen polls that have put John McCain up by 9 percentage points in the Old Dominion. We've also seen polls that have Barack Obama up by 9 percentage points.
What gives?
Well, consider this a consumer's guide to presidential polls -- from someone who has both written about polls and helped design them.
- Is this a poll of registered voters or likely voters? A year out from an election, the distinction doesn't really matter. But a month out, it's a big deal. Different pollsters have different ways to gauge whether someone is "likely" to vote. Sometimes there's a difference between the two groups, sometimes there isn't. At this stage, though, you probably want to know about "likely" voters.
- When was the poll taken? Recent is always better. But I always look to see how many days it took to conduct the poll. If the poll was "open" for a long time, it is likely less reliable. Too many events can intervene in a week's time to change people's opinion. By the same token, a poll taken on just one day may be too susceptible to that one day's events.
Russia invades Georgia! Better swing to McCain!
Wall Street melts down! Better switch to Obama!
Mice infest the Roanoke City Market Building! Quick -- which candidate has the better plan, or at least the better mousetrap?
Come November, the whole shebang will be settled with a one-day "poll" -- but at least then, some 60 percent, 70 percent, maybe 80 percent of the voters will take part. Until then, I like to see a poll conducted over two or three days.
- Was this a survey taken by an actual caller -- or an automated machine? Survey USA uses robo-calls; Rasmussen also does some robo-calling. Some think they're not as accurate as actual interviewers, others say there's no evidence to support that. I always put an asterisk beside those, and if they're out of whack with other polls, I begin to doubt them. (One poll that showed Obama with a 6-point lead in Virginia was a robo-call poll, by the way. But the CNN/Time poll that showed him up by 9 points was not.)
- The size of the polling sample doesn't really matter -- assuming it's an actual probability sample, which is a fancy statistical term meaning, "Well, it's a 'good' sample." If it is, the rest is just math, and you can look at the margin of error to decide how much the results might be off. But don't get hung up on a 1,000-person sample being better than 600-person sample. If the pollsters have done their statistical homework, it's not a big deal.
- What you really want to know, however, is what assumptions the pollster made about who will vote -- and often that's difficult to discern. Take the Christopher Newport University poll that showed McCain with a 9 percentage point lead in Virginia -- far different from any of the other polls in circulation.
Dig deeper, though, and you find out why. The pollster assumed that blacks would constitute only 12 percent of the voters on Election Day, and weighted the results accordingly. Because Virginia has a population that's approximately 20 percent black, and because blacks tend to vote Democratic by overwhelming numbers, the effect of this poll was to depress Obama's numbers.
By the same token, I've seen other polls that assumed black turnout in Virginia this November will constitute 22 percent of the electorate -- and those polls may overstate Obama's numbers.
What's the right number to use? You tell me. But in 1989, when Doug Wilder was running to become the state's first black governor, black voters accounted for 17 percent of the total, according to the Weldon Cooper Center for Public Service at the University of Virginia.
The same problem exists with guessing the rest of the electorate. What percentage will evangelicals constitute? What percentage will voters under 30 constitute? And so forth and so on.
A polling model is a lot like cooking without a recipe. The cook knows what the ingredients ought to be but has to make an educated guess at their proportions. Too much of this or too little of that, and the whole thing falls flat.
And just keep in mind another old truism: Ultimately, the poll that matters is Nov. 4.
Dwayne Yancey is senior editor for new channels at The Roanoke Times.
