AT THE EVE of the general election, many people pay special attention to opinion polls. Without exception, these are reported in the news, where journalists and pundits fervently try to explain small movements in the parties’ fortunes.
After the votes are counted, however, often the same people complain about the differences between the opinion polls and the outcome of the election.
This makes that polls are both loved and loathed.
The big issue is that users often do not understand what polls do and don’t tell us. The truth is that these are useful tools to gain insights in public opinion, but as they are based on samples we need to take the margins of errors into account.
Opinion poll reporting in Ireland would greatly benefit if media and readers would pay attention to a few simple lessons.
1. Don’t try to explain every tiny movement in the polls as ‘real change’. The so-called ‘margin of error’ means that small changes in party support are likely to be the result in selecting a different sample of people for each poll.
The exact margin of error will depend on the size of the poll as well as the size of the party: for larger parties, the margin of error will be larger than for smaller parties.
As a rule of thumb, for larger parties changes of 3% of less might not represent true change in the party’s support.
2. Don’t ignore your own disclaimer. All Irish pollsters report their error margins and these are mostly reported in news stories’ disclaimers. But at the same time headlines will prominently state that ‘Labour is losing support’, while according to the very disclaimer in the article that change is not statistically significant.
3. Don’t think that polls will predict elections exactly. If they did, no state would organise elections anymore. The most recent polls were done over the weekend and at that time a significant share of (potential) voters had not decided who to vote for.
Moreover, it is not easy to obtain a good, random sample, of voters, which leads to polling misses, like the one in the UK last year. Last time most Irish polls were quite close to the final result, but that is no absolute guarantee for a repeat.
1. Do look at multiple polls at the same time. All of these polls aim to measure the same thing, so it is strange to ignore other polls than the one your newspaper or outlet is sponsoring.
Polls-of-polls take this idea to the next level, by aggregating multiple polls into one estimate. Michael Marsh’s poll-of-polls for RTÉ and my own Irish Polling Indicator are good examples of this.
2. Use polls to look at trends, not short-term changes. Because of the uncertainty associated with polls, they have a hard time measuring small changes in the short term. Their strength lies in identifying trends over the slightly longer term. If a new poll comes out with surprising news, think “it’s only one poll” and wait for confirmation.
3. Take into account ‘house effects’: structural differences between pollsters. Polling companies in Ireland use different methods to conduct a poll: face-to-face interviews or by telephone, using different techniques to select participants and to correct (or not) for likely voters.
This results in structural differences between pollsters. Red C, for example, usually estimate Labour a bit higher than other pollsters, while Millward Brown and Ipsos MRBI are more positive about Sinn Féins support. While it is not always clear who is right, at least we should be aware of these differences.
If you want to take polls seriously, know their strengths and limitations. If done well, they do provide valuable insights into what people think and how they would vote.
Tom Louwerse is Assistant Professor in Political Science, Leiden University, the Netherlands. He created the Irish Polling Indicator while working at Trinity College Dublin. This tool combines all opinion polls to arrive at a common estimate of party support in Ireland.
To embed this post, copy the code below on your site
have your say