A Peek Behind the Curtain of BC Conservative Leadership Polling

As the Conservative Party of BC leadership race enters its last stages, two experts caution against relying on any of the polling results so far to predict the outcome.

The deadline for signing up new members eligible to vote passed April 18, and the five candidates will try to win over members ahead of the May 30 vote.

The five are former BC Liberal cabinet minister Iain Black, former BC United party vice-president Caroline Elliott, former Conservative member of Parliament Kerry-Lynne Findlay, entrepreneur Yuri Fulmer and former Kamloops mayor and current Conservative MLA Peter Milobar.

During the campaign various candidates have released polling results they claim show they are ahead or at least competitive.

But DJ Sandhu, who taught for four decades at universities including Simon Fraser University and the University of the Fraser Valley, said past polls should be viewed skeptically.

“Take it with a grain of salt, for whatever it’s worth, and do your own due diligence,” he said.

There was a debate Friday and two more are scheduled.

A Conservative member who did some work on Peter Milobar’s campaign in its earlier stages, Sandhu warned that campaigns can use dubious poll results to shape a narrative and people’s opinions.

“This is, again, the perpetuation of misinformation, perhaps not disinformation, but maybe even that,” he said.

He was particularly critical of polls that introduced “order bias” by failing to present names alphabetically as they will appear on the actual ballot.

For example, a poll that Mainstreet Research did for Milobar’s campaign listed Milobar first on a question about who respondents believed was best to unify the party, Sandhu said. “There’s clearly an order bias, and that’s well documented in the research field, people will have the tendency to pick the first one.”

Only one of the surveys Sandhu had seen, the one done for Black’s campaign, listed the names alphabetically, and that may have been because it favoured Black. In the other polls, he said, “they are conveniently listing them the way they want to list them.”

Other problems included asking double-barrelled questions, for example, combining categories like “jobs and economy” together.

“Those are two different things,” Sandhu said. “When somebody is responding favourably or unfavourably, which one is it? That’s when [people interpreting the results] pick what they want to pick.”

Some surveys offered choices that were not mutually exclusive, he said, such as asking how long someone had been a party member and giving choices of “three to six months” and “six months to two years.”

“If I’ve been a member for six months,” he asked, “which one do I check?”

One survey asked respondents to rank three different candidates on 60 different issues. “People won’t have the patience to do that,” he said. “Now you have to think about the quality of the data, and they’ll push out whatever they want to push out.”

There’s also a “glaring” issue with using interactive voice response polling, he said, which involves making automatic calls to a list and taking responses from whoever answers. “It’s basically if you shoot enough shot, you’re bound to hit something. How many calls did they make to get the 1,054?”

Nor did Sandhu approve of assigning a margin of error to surveys that didn’t follow standard research methods. “Unless it’s a proper random sample, they should not even be mentioning that. That’s only put there to perhaps maximize the ‘rub-off’ effect and try to establish credibility.”

Mainstreet Research CEO Quito Maggi mostly disagreed with Sandhu’s criticisms but identified another reason to be strongly skeptical about the polls the campaigns have released.

Mainstreet deals with order bias by always randomizing the order options are presented in, he said, especially when asking about voter intent.

As for interactive voice response, Maggi said, the firm also surveys by text and email. In the case of the Conservative Party of BC, the membership is relatively homogeneous and any sample of it is likely to be representative.

“It’s old, it’s white and it’s male,” he said. “It’s not like the general population of British Columbia. It’s very, very different.”

Maggi also said that Mainstreet is very clear in its reporting of results that it can’t assign a margin of error when the sample is coming from a subset of the general population such as a party membership, though they do include it for comparison.

The way it was reported on one Mainstreet survey for the Milobar campaign was, “While a random sample of a non-random population like a membership does not carry a traditional margin of error, for comparison, a random probability sample of this size would carry a margin of error of +/-1.93 19 times out of 20.”

It’s true that sometimes campaigns will be selective in what they report, Maggi said. “I hate when clients or other third parties, lots of times whether it’s interests groups or people we do polls for that leak them and then sort of mischaracterize some of the findings for their own purposes, but that’s the nature of our business sometimes.”

Maggi did see another big reason to distrust the early polls on the Conservative Party of BC leadership race.

“It’s the nature of the lists that each campaign got,” he said. Each one had the list of members who had signed up through the party’s website, plus the ones that had come in through their own portal. Until the sign-up period ended, none of them had the whole list.

“That’s why all the polls are very different, because everybody is polling a different list,” Maggi said. “Of course you’re going to end up with these very disparate results from all the different campaigns internally, until now.”

When Mainstreet polled for Milobar’s campaign, it had a list of no more than 16,000 members. Now, according to the party, there are 42,000.

“That universe has completely changed, and we really don’t know what that universe looks like,” said Maggi, who added that he would like to survey the list but so far nobody has asked him to.

It also means that the numbers released earlier in the campaign are unlikely to be anywhere close to the final results.

“This isn’t deliberate,” Maggi said. “No pollster is going to put their thumb on the scale, but the fact that the list that they’re given inherently is already biased toward the candidate that they’re polling for, means what it means.”

Early in the race, the campaign teams have incentive to generate and release any numbers that are favourable to their candidates, he said. “I think it’s about trying to show momentum. There’s a myriad of reasons.”

Maggi said it’s unlikely that any of the campaigns will release internal numbers between now and voting day and that whoever is in the lead is unlikely to be the eventual winner.

The leadership contest uses a ranked ballot, which means eligible members can rank the candidates. There is also a point system where each of the province’s constituencies is given a maximum of 100 points.

If nobody receives enough votes to take more than half the points on the first count, the candidate with the fewest points is dropped. Their supporters’ second choices are distributed to the remaining candidates. That continues until someone wins.

Front-runners tend to become polarizing candidates, which can be an issue on a preferential ballot, Maggi said, adding that unless they can cross the 30 per cent threshold on the first ballot, they are likely to be passed in later rounds by other candidates.

It is usually the “least objectionable” contestant who ends up moving to the front, Maggi said. “The one who can appeal most broadly across most of the ideological spectrum within a membership is the one that can win.”

It will come down to turnout, regional weightings and the points value of votes, he said. The debates will matter a great deal as candidates try to establish themselves as a solid second or third choice for supporters of their opponents.

Anyone who survives the first round of voting could ultimately come out on top, Maggi said. “Almost all five candidates have a chance to win.”