The article you just read was brought to you by a few thousand dedicated readers. Will you join them?

Thanks for coming by The Tyee and reading one of many original articles we’ll post today. Our team works hard to publish in-depth stories on topics that matter on a daily basis. Our motto is: No junk. Just good journalism.

Just as we care about the quality of our reporting, we care about making our stories accessible to all who want to read them and provide a pleasant reading experience. No intrusive ads to distract you. No paywall locking you out of an article you want to read. No clickbait to trick you into reading a sensational article.

There’s a reason why our site is unique and why we don’t have to rely on those tactics — our Tyee Builders program. Tyee Builders are readers who chip in a bit of money each month (or one-time) to our editorial budget. This amazing program allows us to pay our writers fairly, keep our focus on quality over quantity of articles, and provide a pleasant reading experience for those who visit our site.

In the past year, we’ve been able to double our staff team and boost our reporting. We invest all of the revenue we receive into producing more and better journalism. We want to keep growing, but we need your support to do it.

Fewer than 1 in 100 of our average monthly readers are signed up to Tyee Builders. If we reach 1% of our readers signing up to be Tyee Builders, we could continue to grow and do even more.

If you appreciate what The Tyee publishes and want to help us do more, please sign up to be a Tyee Builder today. You pick the amount, and you can cancel any time.

Support our growing independent newsroom and join Tyee Builders today.
Canada needs more independent media. And independent media needs you.

Did you know that most news organizations in Canada are owned by just a handful of companies? And that these companies have been shutting down newsrooms and laying off reporters continually over the past few decades?

Fact-based, credible journalism is essential to our democracy. Unlike many other newsrooms across the country, The Tyee’s independent newsroom is stable and growing.

How are we able to do this? The Tyee Builder program. Tyee Builders are readers who chip into our editorial budget so that we can keep doing what we do best: fact-based, in-depth reporting on issues that matter to our readers. No paywall. No junk. Just good journalism.

Fewer than 1 in 100 of our average monthly readers are signed up to be Tyee Builders. If we reach 1% of our readers signing up to be Tyee Builders, we could continue to grow and do even more.

If you appreciate what The Tyee publishes and want to help us do more, please sign up to be a Tyee Builder today. You pick the amount, and you can cancel any time.

Support our growing independent newsroom and join Tyee Builders today.
We value: Our readers.
Our independence. Our region.
The power of real journalism.
We're reader supported.
Get our newsletter free.
Help pay for our reporting.
Mediacheck

Hey, Careful Where You Swing that Poll!

More muddled election math.

By Tom Barrett 6 Dec 2005 | TheTyee.ca
image atom

If there were no real winners in the last federal election, there was one big group of losers: the pollsters.

At the end of a campaign that saw an unprecedented emphasis on media horse-race polls, the pollsters all had the Liberals and Conservatives in a dead heat. Seat projections based on the pollsters' numbers had most of the media predicting a Conservative minority government.

Instead, the Liberals beat the Conservatives by more than seven percentage points - a result that fell well outside all the major pollsters' margins of error. The pollsters argued that their numbers had been right but that the voters, especially in Ontario, had changed their minds at the last minute, after the last polls had been taken.

They were probably right. We might have hoped, though, for a little more caution in the interpretation of those last polls, given everything the pollsters have said since about the volatility of the voters.

'Aggressive' polling

But whatever the reason for the red faces in 2004, you might think that the pollsters - and the media that breathlessly reported their findings during the campaign - would vow to be more careful this time.

To a certain extent, they are. Pollsters are promising to stay in the field later this time around. They're interviewing more people. The Globe and Mail, for example, is promising bigger and better polls this time, what editor-in-chief Edward Greenspon calls "our most aggressive ever public-opinion program."

But anyone who read last Wednesday's Globe might be left wishing for a little more care and a little less aggression.

Under the headline "Liberals losing allure among minorities, poll suggests," the Globe described how immigrants and visible minorities appeared to be switching to the Conservatives.

The 20-column-inch story, which was accompanied by three pie charts, talked about how "the Liberal Party's traditional stranglehold on support from immigrants and visible minorities may be vulnerable to a sustained Conservative assault focusing on government corruption…"

This was based on a finding in a Strategic Counsel poll that the Conservatives were supported by "just under 30 percent" of visible minority respondents. (According to one of the pie charts that accompanied the article, the figure was 27 percent.) A further 38 percent of visible minority respondents supported the Liberals, placing the Conservatives "within striking distance" among this demographic.

The story didn't say what previous polls had reported about visible minority and immigrant voting intentions, but it did suggest that this was a sizeable gain for the Conservatives.

Never mind

Various reasons were given why minority groups might be abandoning the Liberals: same-sex marriage, the sponsorship scandal and Conservative organizing efforts.

Various experts were interviewed to speculate about the results. The recent B.C. municipal elections were factored in. The attitude of the Bloc Quebecois towards visible minorities - and vice-versa - was assessed.

Then, in the second-to-last paragraph of the story, the Globe ran into an expert who knew something about polling. Richard Johnston, the head of the UBC political science department "cautioned against reading too much into the Strategic Counsel poll because the statistical sample of minorities is necessarily smaller than the larger survey sample."

"Indeed," the story continued, the Strategic Counsel had noted that the stuff about visible minorities "was added as an area of interest only, and is not considered a scientific poll on par with the larger survey."

In the words of Saturday Night Live Weekend Update correspondent Emily Litella, "Never mind."

Turns out the survey in question interviewed only 180 visible minority persons across the country. That means that the sampling error for any result from this group would be greater than plus or minus seven percentage points, 19 times out of 20.

At the outside limits of the sampling error, the Conservative support among visible minorities could be as low as 20 percent and the Liberal support could be as high as 45 percent. Or the Conservatives might be ahead. You don't know.

The fact is, a national sample of 180 just isn't going to tell you much of anything.

Intrigue without substance

The point here is not to dump on the reporter in question, or on The Globe. Lots of media make similar mistakes; at least this story admitted at the end that the numbers weren't particularly trustworthy. (Although they were described as "intriguing.")

The point is that, as a news consumer, you shouldn't put too much stock in what national polls say about who Nova Scotians plan to vote for, or what Green Party voters think about private health care.

The media make plenty of mistakes when it comes to interpreting polls. Giving too much weight to the results of small groups within the overall sample is one of the most common.

Tom Barrett is a contributing editor to The Tyee.  [Tyee]

Share this article

The Tyee is supported by readers like you

Join us and grow independent media in Canada

Facts matter. Get The Tyee's in-depth journalism delivered to your inbox for free.

LATEST STORIES

The Barometer

Tyee Poll: What Is One Art or Design Skill You Wish to Learn?

Take this week's poll