Opinion polls, surveys and studiesHow to report them.
Opinion polls have become very common in the media. However, reporters should be aware that many of the online polls carried out by Web sites and polls taken through SMS messages have no scientific validity.
For an opinion poll to be valid, it should satisfy several criteria such as sample size and scientific selection of the sample. One should be vary of poll results if sample size is very small and if the sample does not cover all sections of the society who are likely to hold divergent opinions on a given question.
Internet polls and SMS polls often fail on both counts. The samples are usually very small and cover only a segment of the society— the Internet users, who often belong to the affluent sections of the society in countries like India. A Web based poll on a subject that directly concerns the Internet users such as, say, the official control of the Web, may have some validity. In other cases, the sample would miss out on predominant opinion in the society. Similar is the case with polls using the SMS facility in mobile phones.
Moreover, such polls are open to manipulation both by the organisers and the respondents. Web based opinion polls usually take precaution to take multiple votes from the same computer (IP address). However, hackers and geeks can often bypass these restrictions. For example, a person would be able to cast a number of voters using proxy machines in different parts of the World.
When reporting opinion polls and surveys, reporters should mention the manner in which the survey is done (through questionnaire, interviews—direct or by telephone, Web, SMS and the like). As mentioned earlier, Web based and SMS based surveys are often not worth reporting.
A random sampling may suffice for many studies and surveys. However, scientific sampling based on population profiles (age, sex, employment status, income levels etc) may give better results. Consult a statistician to see whether a survey is properly done. Opinion polls about the outcome of an election should cover a wide sample and people from all walks of life, age groups, and strata of society.
The agency that did the study and its credibility, interests and affiliations are also important in assessing the veracity of a study. Funding for some opinion polls could be from interested parties. Reporters should cross check on it and mention that in their reports. For example, a study done by a tobacco company about incidence of cancer should be taken with a pinch of salt. It is not uncommon for lobbyists to use opinion polls to influence decision makers. Poll results released through Internet require scrutiny as the actual sponsors may be hiding behind some front organisation. Better not to pick up results of such polls and surveys for your report. Studies get published in scientific and professional journals only after peer review.
Margin of error is another factor to be considered and included in reports. Any scientifically done survey or study would be able to report its margin of error. A change reported within the margin of error is not statistically significant. For example, if the popularity rating of a political personality is found to have increased from 49 to 51 per cent with a margin of error of four percent, the results do not show that the majority now supports him. His support could even be 47 per cent at the time of the survey. In other words, an increase of two percentage points may not mean an increase when the error margin is higher than that.