Digging into Last Week's Blitz of Polls

Some random thoughts on a flurry of election polls...

Five of the six polls that were released last week suggest the gap between the New Democratic Party and the BC Liberals has narrowed substantially. At the end of the week, B.C.'s two biggest pollsters, Angus Reid and Ipsos Reid, weighed in. Ipsos found a 10-point gap, Angus Reid found a seven-point gap.

As you can see from the chart here, Reid shows the BC Liberals up three points from the beginning of the campaign and the NDP down four points. Ipsos shows the Liberals up six points from their start-of-the-campaign poll and the NDP down three points. Even that six-point Liberal jump from Ipsos is within the poll's combined stated margins of error -- although it's near the outside edge.

But a trend is apparent. The polls are tightening -- just as pretty much everyone predicted at the beginning of the campaign.

Is the shift due to last week's televised leaders debate? Could be. Or it could be a result of negative Liberal ads or the relentless, if fanciful, Liberal focus on debt, the deficit and the economy during the first two weeks. Or the media's focus on the same topics and on NDP leader Adrian Dix's Kinder Morgan switch.

Or, quite likely, a lot of potential voters are just starting to pay attention to politics for the first time in four years.

We'll never know the real reason because there isn't enough information available.

Forum's four point gap: sample skewed?

Not that there's a shortage of facts and figures flying around from the week's polls. Take the Forum poll that suggested the gap between the NDP and the Liberals had fallen to four points.

Here's something interesting about that poll that my colleague Andrew MacLeod pointed out: out of 1,009 respondents, 459 said they voted for the Liberals in the last provincial election. Another 290 said they voted for the NDP, 78 said they voted BC Conservative, 79 said they voted Green, 49 said they voted for other parties and 54 said they didn't vote.

Forum is to be commended for publicizing this much detail about their poll; not all pollsters do.

But among those respondents who say they voted in 2009, 48 per cent say they voted for the Liberals, 30 per cent for the NDP, eight per cent for the Conservatives, eight per cent for the Greens and five per cent for other parties. Five per cent of the total sample say they didn't vote.

Here are the results from the 2009 election: Liberal, 46 per cent; NDP, 42 per cent; Conservative, two per cent, Green, eight per cent; others, two per cent.

And an estimated 49 per cent of all eligible voters didn't vote in 2009.

Questions like this rely on sometimes faulty memories. Respondents who didn't vote may feel pressured to lie; saying you voted is the socially acceptable answer, after all. And pollsters who discover their sample doesn't look like the population use weighting techniques to make up for the difference.

Still, on the face of it, this sample doesn't look much like the real world of B.C. voters.

Gender gap trap?

One interesting difference between the Ipsos and Reid polls involves the gender gap. Ipsos suggests women favour the NDP by a 20-point margin, 50 per cent to the 30 per cent who would vote Liberal. But Reid suggests the gap is eight points, 43 per cent NDP to 35 per cent Liberal. That's a pretty big difference, but it's worth remembering that the margin of error goes up as the sample size goes down.

In this case, we're looking at the sample of women only, which will be half of our overall sample. That means that in a poll of 800 prospective voters, our gender gap numbers will be based on a sample about 400 women -- which would have a margin or error of about five points, plus or minus, 19 times out of 20.

This goes for regional samples, as well, which is why it's not a good idea to try to predict what will happen in Victoria, say, from a province-wide poll. A B.C. poll of 800 people will have about 400 respondents in Metro Vancouver, and could have fewer than 150 on Vancouver Island. That's a margin of error of around plus or minus eight points for the Island.

(That's assuming you believe it's appropriate to quote a margin of error for online polls like these Reid and Ipsos surveys. Some experts say you shouldn't; Reid quotes standard margin of error figures, Ipsos uses something called credibility intervals. The main thing to keep in mind is that, all else being equal, smaller samples tend to be less reliable.)

Will polls change decisions at the polls?

One further question that's been debated a lot in the last week is the impact these polls will have on voters.

Wilfrid Laurier University political scientist Jason Roy studies the question of whether polls change anyone's vote. The answer, he said in a recent interview, is maybe.

Polls can motivate supporters to work harder for a party or candidate that seems to be trailing. Or they can motivate people to stay at home on election day because they figure their candidate is a shoo-in. There's a theory of a bandwagon effect -- that voters flock to the party that seems to be winning. And there's a theory of the underdog effect -- people back the party that's trailing.

In Canada, there doesn't seem to be any evidence to support either theory, Roy said. It's possible that the two effects cancel each other out, he said. And the impact of a poll could depend on the size of the gap between the two parties.

"There's a whole lot of factors here that are largely a function of the individual," said Roy.