Mediacheck

Globe Defends Polling Techniques

Number, content of questions without bias says paper's managing editor.

By Monte Paulsen 17 May 2005 | TheTyee.ca

Monte Paulsen is a certified energy advisor. He helps homeowners and homebuilders save money and cut carbon emissions. He works in collaboration with City Green Solutions.

Monte wrote for The Tyee between 2005 and 2010. While at The Tyee he reported on affordable housing and green building. Links to his work can be found below.

He has also written for numerous newspapers and magazines including Canadian Geographic, Vancouver Magazine and The Walrus.

And he is a co-author of Beyond the Deep: The Deadly Descent into the World's Most Treacherous Cave (Warner Books, 2002).

Email Monte or follow him on Twitter.

image atom

The Globe and Mail has defended last week’s Strategic Counsel poll, which was questioned by polling experts in a Tyee article.

The Globe poll predicted that the BC Liberal Party would beat the New Democrats by a 13 per cent margin, while local polls conducted within days found margins of 5 and 8 per cent.

“With four days to go until the election,” The Globe and Mail reported on the front page of its BC section on Friday the 13th, “the governing BC Liberals have surged into a commanding lead that will produce another solid majority for the party.”

A May 16 story in The Tyee quoted experts who found the survey questioning technique unorthodox. The Globe poll asked 14 questions before inquiring how respondents planned to vote. Vancouver pollster Angus McAllister examined the sequence of questions and told The Tyee the approach could be “stimulating a thinking process that was not already there.”

McAllister questioned whether the number, sequence and content of the questions might introduce bias, and whether it was appropriate to release the results of such a polling technique so late in an election campaign.

“If you’re really trying to find out where people stand, you don’t want to ask them questions that will lead them in any one direction,” observed Steven Rosell, a partner to highly respected American pollster Dan Yankelovich, in the Tyee story.

Globe’s emailed response

Tyee editor David Beers emailed a list of related questions to Globe editor-in-chief Edward Greenspon. Beers received this email yesterday from Globe managing editor Colin MacKenzie:

Mr. Beers: I respond to your questions to Edward Greenspon below, but let me by way of prologue, say that The Globe and Mail takes very seriously its role in the national conversation and has been a world leader over the last 20 years in marrying survey research with journalism. We take great exception to the tone and comments in your web posting that imply that we distorted our journalism in pursuit of the interests of any party beyond our readers.

Here are the facts: we, in partnership with CTV, commissioned two exclusive polls for the B.C. election from Strategic Counsel.

The first ran just as the campaign started and showed the Liberals with an eight-point lead. The second, as has been customary with our election polling for decades, was scheduled to run the week before the vote. Strategic Counsel, one of Canada's pre-eminent research organizations, designed both polls in consultation with senior editors at the Globe and CTV.

Angus MacAllister's huffing and puffing in the piece would actually have some methodological merit if it did not apply to the questionnaire we administered. The bias associated with "structuring" and "framing" effects occur when preceding questions are argumentative or biased in one direction, only. Our questions, in every instance, give respondents an equal opportunity to answer in the affirmative or negative and therefore to support either the Liberal or NDP position.

By posing the questions we are trying to more closely approximate the dynamic that will occur in voters' minds in the final days before election day.

There was no 'message testing' involved, or it goes without saying,. any intent to do anything but report what British Columbians are thinking in the week before an election. All of Strategic Counsel, The Globe and Mail and CTV have a long history and great credibility about polling. It is outrageous to suggest we may be favouring one side or another in our news pages. It is so outrageous that one wonders about the motives of those make the charges.

One wonders if they simply didn't like the result. This appears especially ironic given that Will McMartin in his Battleground B.C. Post yesterday had this to say: ‘Six province-wide polls two each by The Mustel Group, Ipsos-Reid and Strategic Counsel -- have been published since the election got under way. They have been remarkably consistent, showing the Liberals with support from 45-49% of the B.C. electorate, the NDP at 36-40%, and the Greens at 10-13%.’

As to some of your questions. After eight years with first Angus Reid and then Ipsos Reid, we decided six weeks ago to move to Strategic Counsel as our pollster of record. They do our national polling as well as regional work.

You will note that we do publish other polls: last Wednesday, we published the Mustel Group poll. (I should note that our sample size was twice as large as the Mustel survey.) As to the matter of the missing question 14, there was a numbering error as we tweaked the poll in its final stages, and there was in fact no question 14. Just to cover off some of your remaining questions. The survey questionnaire was developed through consultation between Strategic Counsel and editors at CTV and the Globe.

The Globe and CTV paid for the work. “It is standard journalistic practice to publish as close to an election date as possible. The literature, as you probably know, is split on the issue of any bandwagon effect for a leading party versus the galvanizing effect on that in second place, a phenomenon that conceivably played out last year federally.

I hope this answers your questions and you will give the response equal prominence in your follow-up coverage.

yours sincerely,

Colin MacKenzie managing editor, news

p.s. ed [Greenspon] asked me to pass on his disappointment about this story. You know better about the Globe's integrity, he says, and he chooses to focus on your kind words of a couple of weeks ago about the Globe's B.C. initiative.

Unorthodox say two BC pollsters

The Tyee spoke with two more of Vancouver’s top political pollsters on Monday. Both backed McAllister’s assertion that the standard practice is to determine voter intention at the outset of a poll.

“Every poll I’ve ever worked on asked the vote question within the first three questions,” said Kyle Braid, a vice president at Ipsos-Reid. “You want to make sure none of the questions you ask introduce any potential bias into the voters’ preferences.

“We will ask one or two very general questions beforehand,” continued Braid. “We would not introduce a topic for discussion, however.”

Ipsos-Reids May 8-10 poll suggested the BC Liberals will win 49 per cent of the popular vote, providing an 8-point lead over the NDP. That poll asked about voter intention in its second and third questions.

Vancouver pollster Evi Mustel agreed. “We always ask [voter intention] in the second question,” said Mustel, whose May 5-9 poll predicted the BC Liberals will win by 5 points. “We ask one warm-up question: ‘What is the most important issue facing BC today?’ We intentionally wrote that question to be open-ended. This way we get the respondent thinking without providing any influence whatsoever.” “It’s not the way I would construct a questionnaire,” Mustel added. “But this is an art and not a science. We all do things differently.”

The Strategic Counsel’s Alan Gregg once again did not return calls and an e-mail from The Tyee.

Monte Paulsen is managing editor of The Tyee’s Election Central.  [Tyee]

Share this article

The Tyee is supported by readers like you

Join us and grow independent media in Canada

Get The Tyee in your inbox

LATEST STORIES

The Barometer

Have any major issues been missed in the Alberta election campaign?

Take this week's poll