Making Sense of Week's Wild Poll Shifts

At the beginning of this week, a poll suggested the NDP was leading the BC Liberals by 22 points. Then another poll suggested a 10-point NDP lead. Then a third poll suggested four points. Then another suggested eight points. Then another one suggested seven.

Given that information, which of the following statements best matches your view:

I believe all these results.
I believe none of these results.
There's too damn many polls.

On Thursday, Forum Research released a poll that suggested the New Democrats' lead over the governing BC Liberals has fallen to four points. The Forum poll was followed later in the day by one from Insights West that suggested the NDP leads by eight points among decided voters. Then Angus Reid weighed in with a poll that suggested a seven-point NDP lead.

So what's a voter to think?

For a start, most of these results fall within the range of the polls' margins of error. The one outlier is the 22-point Justason poll, which was conducted April 15-23 -- about a week before the other results. Things can change quickly during an election campaign; the dates a poll was taken are a key factor.

To see the Tyee's table showing 17 polls going back to mid-January of this year, click on this Election Hook item published late last evening.

Next time you read a poll, arm yourself

Everyone has their own poll, it seems, and there often seems to be precious little agreement among them. Here are some resources that can help the average political junkie make sense out of all these numbers.

The National Post: What the #!%*?: Election polling

This helpful article was written during the October 2011 Ontario election, when poll results were – surprise – bouncing all over the place. Reporter Tamsin McMahon talks to the experts about why this kind of thing happens.

The Globe and Mail: A pollster's painful reckoning: 'How could I have screwed up so badly?'

This article, also from 2011, looks at how EKOS pollster Frank Graves blew his federal election call and what he learned from the experience.

There are a number of good resources that explain how polls work, such as Matthew Mendelsohn and Jason Brent, Understanding Polling Methodology.

This lengthy 2001 article, a Queen's University professor and a pollster, is a bit dated when it comes to the methods pollsters use to contact voters -- phone calls from live operators, automated calls, online panels -- but it contains much helpful information for anyone trying to understand the procedures and pitfalls of polling.

There are also a number of polling resources aimed at journalists. Not all journalists pay attention to the principles they lay out, but an interested news consumer can learn a lot from them. Online stories should contain a link to a poll's methodology. Take a look at the information the pollster makes public and think about the questions provided in these resources.

The Canadian Press Stylebook: Polls

This guide was put together by Canada's national news agency and the Marketing Research and Intelligence Association, a polling industry trade group. "One key factor" for deciding if a poll is valid, this guide states, "is whether the firm belongs to a self-regulatory association" like, um, the MRIA. While membership can be a guide to evaluating a pollster, it should be noted that some reputable Canadian pollsters do not belong to the MRIA for principled reasons.

Sheldon R. Gawiser and G. Evans Witt: 20 Questions A Journalist Should Ask About Poll Results

What it says. Twenty excellent questions to ask about any poll.

The Shorenstein Center at Harvard: Polling fundamentals and concepts: An overview for journalists

It's U.S.-centric, of course, but most of what this respected institute offers applies in Canada as well.

The Roper Center: Fundamentals of Polling

A tutorial from a U.S. pollster.

One hot topic in polling involves, as we suggested above, the different methods being used. Some pollsters use telephone banks with live operators to call potential respondents. Others use an automated system called Interactive Voice Technology, sometimes known by the unflattering term of "robocalling." Other pollsters draw their respondents by email from panels of people who have volunteered to be interviewed.

All these modes raise issues. Nate Silver wrote about the issues in a New York Times article headlined "Before Citing a Poll, Read the Fine Print".

The Atlantic ran a different view under the headline "A More Perfect Poll."

Finally, here are articles from The Tyee's archives that try to make sense of this stuff.

The 'Dirty Secrets' of Pollsters The most poll-driven election in memory has media and voters mesmerized. Don't be fooled, warns Angus Reid.

Who's Ahead in BC, Libs or NDP? Making sense of conflicting poll numbers and methods.

Pollsters in High Stakes Race, Too The numbers are all over the place. Who will get to brag about being right?

Polls 'Don't Predict the Future' And more hard truths about the use and abuse of modern opinion research.