The article you just read was brought to you by a few thousand dedicated readers. Will you join them?

Thanks for coming by The Tyee and reading one of many original articles we’ll post today. Our team works hard to publish in-depth stories on topics that matter on a daily basis. Our motto is: No junk. Just good journalism.

Just as we care about the quality of our reporting, we care about making our stories accessible to all who want to read them and provide a pleasant reading experience. No intrusive ads to distract you. No paywall locking you out of an article you want to read. No clickbait to trick you into reading a sensational article.

There’s a reason why our site is unique and why we don’t have to rely on those tactics — our Tyee Builders program. Tyee Builders are readers who chip in a bit of money each month (or one-time) to our editorial budget. This amazing program allows us to pay our writers fairly, keep our focus on quality over quantity of articles, and provide a pleasant reading experience for those who visit our site.

In the past year, we’ve been able to double our staff team and boost our reporting. We invest all of the revenue we receive into producing more and better journalism. We want to keep growing, but we need your support to do it.

Fewer than 1 in 100 of our average monthly readers are signed up to Tyee Builders. If we reach 1% of our readers signing up to be Tyee Builders, we could continue to grow and do even more.

If you appreciate what The Tyee publishes and want to help us do more, please sign up to be a Tyee Builder today. You pick the amount, and you can cancel any time.

Support our growing independent newsroom and join Tyee Builders today.
Canada needs more independent media. And independent media needs you.

Did you know that most news organizations in Canada are owned by just a handful of companies? And that these companies have been shutting down newsrooms and laying off reporters continually over the past few decades?

Fact-based, credible journalism is essential to our democracy. Unlike many other newsrooms across the country, The Tyee’s independent newsroom is stable and growing.

How are we able to do this? The Tyee Builder program. Tyee Builders are readers who chip into our editorial budget so that we can keep doing what we do best: fact-based, in-depth reporting on issues that matter to our readers. No paywall. No junk. Just good journalism.

Fewer than 1 in 100 of our average monthly readers are signed up to be Tyee Builders. If we reach 1% of our readers signing up to be Tyee Builders, we could continue to grow and do even more.

If you appreciate what The Tyee publishes and want to help us do more, please sign up to be a Tyee Builder today. You pick the amount, and you can cancel any time.

Support our growing independent newsroom and join Tyee Builders today.
We value: Our readers.
Our independence. Our region.
The power of real journalism.
We're reader supported.
Get our newsletter free.
Help pay for our reporting.

The Weight on a BC Pollster's Shoulders

Many flubbed their May 14 election calls. Here's one who thinks he knows why.

By Tom Barrett 21 Jun 2013 |

Find Tyee contributing editor Tom Barrett's previous Tyee articles here. Find him on Twitter or email him.

image atom
Ipsos Reid's Kyle Braid unravels the mystery of his election-night failure. Photo: Twitter.

Pollster Kyle Braid thinks he's figured out how he missed so badly in the May 14 election.

Like almost every other pollster, Braid, a vice-president at Ipsos Reid, came up with results on the eve of the election that suggested a healthy victory for the New Democratic Party. When that turned into a healthy victory for the BC Liberals, Braid and the other pollsters were left with egg dripping from their faces.

The last Ipsos poll, taken May 13, suggested the NDP had the support of 45 per cent of decided voters, compared to 37 per cent for the BC Liberals. The final results were NDP 40 per cent, Liberals 44.

Braid told an audience at the Spur Festival in Vancouver last week that he thinks he adjusted his data when he shouldn't have and failed to adjust it when he should have.

Every polling company adjusts, or weights, its samples. That's because samples rarely look like the general population. They may, to take a simple example, contain more women than the electorate as a whole.

We know that women vote differently than men, so, to make such a sample more representative, pollsters will weight the data -- giving a higher value to men in this example.

Such weighting often contains a large dollop of intuition along with the statistical science. Ideally, proper weighting will make a sample match the demographics of the people who actually turn out to vote -- which are not the same as the demographics of the general population.

We know that a person's age, income and education all predict whether they will vote. Older people are a lot more likely to vote than young folks. (See below.)

At the end of the campaign, Ipsos's polls had the Liberals ahead among voters 50 and older, "but we didn't make any adjustments for that," Braid said. "We know that older folks make up about a third of the electorate, but they account for about 50 per cent of the voters."

Ipsos didn't give enough weight to these older voters.

Said Braid: "If we'd just done something as simple as reweighted our results based on age, the Liberals would have gone up immediately two or three points."

The past vote trap

An even bigger problem came when Ipsos weighted its data based on who respondents said they voted for in 2009.

Ipsos's samples tended to be heavy on people who said they voted for the Liberals last time, Braid said. To account for this, the company gave less weight to their opinions.

Not a good idea, as it turned out.

Asking people who they voted for comes with some dangers, Braid said. You're relying on their memories: they might have forgotten how they voted, they might not want to admit they didn't vote, they might not want to admit they voted for the losing party or they might not want to admit they voted for a currently unpopular party.

582px version of Turnout-by-agegroup.jpg
Source: Elections BC

You don't know how many of your respondents fit which of those categories, and that makes weighting by past vote a dangerous practice. Braid said he did it because when Ipsos switched from telephone polling to online polling a few years ago, the online results didn't "match up" with phone results.

But when he weighted by past vote, they did match up, he said.

"I was comfortable with that," Braid said. "I did that with the HST referendum in 2011. Hit the number almost bang on the head and said this is the way to do it."

Part of that false sense of comfort during the May 14 campaign came from the numbers released by other pollsters. "Our results looked like everybody else's," he said. "Nobody... was telling a different story."

One thing Braid forgot, he said, is that further you get from an election, the less reliable past-vote declarations are.

Weighting by past vote "tended to lift the NDP up three [percentage points] and the Liberals down two or three points" in the Ipsos polls, Braid said.

On election day, Ipsos polled people who had just voted. The results line up almost perfectly with the election outcome, Braid said -- but only if you don't weight for respondents' memories of their 2009 vote.

Eleventh hour Liberal shift?

Some pollsters and commentators have speculated that the giant May 14 poll fail was caused at least in part by a huge last-minute shift to the Liberals that took place after the pollsters stopped polling. Braid said exit polls taken on election day suggest that about a quarter of B.C voters made up their minds in the last week of the campaign.

But, it appears, they didn't go en masse to the Liberals.

"They voted a little bit more for the Liberals," Braid said. "Maybe it accounts for one point of the seven points of our miss."

While Braid thinks he's unravelled the mystery of his own election-night failure, other pollsters have told him they don't see the same issues behind their own results.

BC Liberal pollster Dimitri Pantazopoulos, whose internal telephone polls called a Liberal majority, told the Spur Festival audience that he is extremely wary of weighting by anything that relies on a respondent's memory. Weighting should be restricted to "empirical data" such a person's age or gender, he said.

Pantazopoulos recalled polling for Stockwell Day and the Canadian Alliance during a turbulent period in 2002. Day's support at the time was in the single digits. When Pantazopoulos asked respondents who they voted for in the last federal election, those results were in the single digits, too -- even though Day had taken 26 per cent of the vote in the 2000 election.  [Tyee]

Share this article

The Tyee is supported by readers like you

Join us and grow independent media in Canada

Facts matter. Get The Tyee's in-depth journalism delivered to your inbox for free.


The Barometer

Tyee Poll: What Is One Art or Design Skill You Wish to Learn?

Take this week's poll