Archives for March 15, 2010

As I follow what is supposed to be the “war to end all wars” over the health care bill, I see various poll results presented by the media folks. Interestingly enough, they tend to vary in their results. To be specific, some show that Americans favor the bill while others say most Americans are against the bill.  This raises the obvious question of how this is possible.  Naturally, what follows applies to polling in general and not just the health care polls.

First, polls have margins of error. This is the number, expressed as a percentage, by which the sample is likely to differ from the population as a whole. For example, if a poll reports that 53% +/- 4%  of Americans think the existing health care bill should be revised rather than scrapped, the margin of error is 4.2%. So, the actual percentage of Americans who have this view is likely to range from 49-57%.  Obviously enough, a poll of the same population could thus get a result within that range, perhaps 49%. As such, the errors inherent to polling can yield different poll results. Naturally enough, folks with an agenda will tend to pick the polls that match their views.

Second, polls can be affected by the wording of the questions.  For example, a poll that asks “do you favor extending Medicare/Medicaid style benefits to all Americans” will tend to garner more positive responses than “do you favor socialized medicine for all Americans?” As such, the desired poll results can often be generated by creating questions that are slanted in the desired way. So, if someone wants to “show” that Americans favor the bill, then questions that use positive slanting can be employed. If someone wants to “show” that Americans are against the bill, then negatively slanted language can be used.

Third, the order of questions and the context being presented can also impact the results. For example, if a question about the health care bill is preceded by a question about massive budget deficits, then the results will tend to be against health care. But, if a question about the bill is preceded by a carefully crafted question about pre-existing conditions, then the results will tend to favor the bill. As with the wording of the questions, this allows people to load polls to get the results they want.

Fourth, the available answers can be restricted. For example, a question that asks “do you favor passing the bill exactly as it is or starting over” provides only two options when, in fact, there are many alternatives. As such, the poll will not accurately capture the opinions of those being polled.

Fifth, people change their minds over time. So, a poll taken yesterday might reflect what people thought yesterday and this might be rather different from what people thought today.

Sixth, polls can be biased. For example, if Fox News conducts a call in poll, then they will get people who are interested enough to call in (one bias) and will tend to get mostly Fox viewers (another bias). The same sort of bias situation also applies to MSNBC, for those who think I’m singling out Fox.

All of these factors (and others I have not mentioned) allow such differing poll results.

As far as what people should do, the rational thing is not not put your faith in one poll (even one that is properly done). A more sensible approach is to consider the results from as many legitimate polls as possible. Of course, people generally tend to just stick with the poll that confirms their own belief.