Use caution when approaching polls

Party preference numbers have varied

But differences deceiving, experts say


(As published in the Toronto Star, January 21, 2006.)

Not a vote has been counted. The party leaders are still criss-crossing the country in a frantic final bid for support. Campaign volunteers are working the phones, replacing lawn signs and arranging election day transportation for voters who require it.

And yet we already know or at least we think we know that on Monday Stephen Harper's Conservatives will defeat Paul Martin's Liberals, and the NDP's Jack Layton will be the federalist leader holding the balance of power in a minority Parliament.

How have we concluded this? Well, the polls tell us so.

But don't different major polls all seem to have differing results?

Yes and no, says Jeffrey Rosenthal, professor of probability theory at the University of Toronto and a leading expert on polling.

Rosenthal, author of the recent bestseller Struck by Lightning: The Curious World of Probabilities, says "any one poll should always be taken with a grain of salt.

"It can always be a little high or a little low depending on who they happened to phone, who happened to be home and so on," he says, emphasizing that "in a way, the polls are not as far apart as you might think.

"There's an awful lot of polls so ... just by chance every once in a while there's going to be a poll that's off from another poll."

Rosenthal points to the example of a Strategic Counsel poll that appeared in The Globe and Mail this week. (The Toronto Star's pollster is EKOS Research Associates, but Rosenthal mentioned the Globe poll and one by SES Research conducted for CPAC without prompting.)

"There's been a lot of talk in the last couple of days about the Strategic Counsel poll, which had about 42 to 24 for the Conservatives (against the Liberals) compared to the SES poll, which had (the Tories at) just under 37 to about 31.5," he says.

"But even if you start comparing them, so the figure for the Conservatives was about 42 in one poll and about 37 in the other that's a difference of about 5 per cent. But the margin of error in the Strategic Counsel poll was about 2.5 per cent and the SES poll was just over 3 per cent, so if you just add up those two margins of error, you're already over 5 per cent."

In essence, both polls could be saying the same thing, Rosenthal says.

"That's one way that, possibly, could explain the entire difference and still be within the margin of error," he says.

A poll's margin of error depends on the size of the sample. The greater the number of people surveyed the smaller the margin. But even with a massive poll, the margin of error can never be zero.

`They're exciting and fun to read about, and also they have a huge impact' -- Jeffrey Rosenthal, U of T

"Every poll, of course, has inherent randomness in it," Rosenthal says.

"Just like if you flip a coin a whole bunch of times, most of the time you'll get just about half heads, but every once in a while you won't. That's related to the margins of error.

"Usually, the poll's error will actually be less than the margin of error. Nineteen times out of 20 it'll be that accurate, but ... there's a pretty good chance it'll be even closer."

Still, Rosenthal acknowledges poll results during a campaign can sway elections, and points to the Conservatives' soaring fortunes in Quebec, which appear to be an echo of results in other parts of Canada.

"That's a case that if there weren't any polls, the Conservative rise in the rest of the country might have happened anyway, but in Quebec I bet it wouldn't," he says, noting the Tories' surge in Ontario led to Quebecers seeing the party as a viable anti-Liberal alternative to the separatist Bloc Québécois.

"Nobody would be thinking about them except for the fact that they're up in the polls."

That's one reason why Duff Conacher, Democracy Watch co-ordinator, is wary of polls.

"One of the big problems with this election campaign and past campaigns is how the polls are hyped given the small samples and the error rate and the undecided factor," Conacher says.

"No voter should look at a national poll or even a provincial breakout and assume that it has any relevance to what is happening in their riding, and that is the only thing that they should be looking at because their vote only counts in their riding," he says.

Conacher says media outlets routinely deceive the public when they publish and broadcast polling stories.

"All numbers have to be given equal prominence in the reporting of a poll. In most of the reporting, the undecided factor is slotted in at the very end of the article with the error rate. That's misleading," he says.

"The error rate should not be slotted in at the end of a story it should be at the beginning of the story. That amounts to misleading the public and it should be made illegal."

Despite the controversy surrounding public opinion polling, Rosenthal insists they are an important part of democracy.

"They're exciting and fun to read about, and also they have a huge impact on not only how the parties and candidates plan their campaigns ... but they also impact the voters a lot," he says.

"When the polls are not just `Who are you going to vote for?' but `How do you feel about abortion or this or that?' in some sense that's the most direct democracy that we have."

[Jeffrey Rosenthal's Home Page / contact page / Struck by Lightning page / Writing Page]