Evidence of Systematic Bias in 2008 Presidential Polling
(preliminary report)
Leonard Adleman
Department of Computer Science
University of Southern California
Mark Schilling
Department of Mathematics
California State University, Northridge
Abstract
Political polls achieve their results by sampling a small number of potential voters
rather than the population as a whole. This leads to “sampling error” which most
polling agencies dutifully report. But factors such as nonrepresentative samples,
question wording and nonresponse can produce non-sampling errors. While
pollsters are aware of such errors, they are difficult to quantify and seldom
reported. When a polling agency, whether by intention or not, produces results
with non-sampling errors that systematically favor one candidate over another,
then that agency’s poll is biased. We analyzed polling data for the (on-going)
2008 Presidential race, and though our methods do not allow us to identify which
agencies’ polls are biased, they do provide significant evidence that some
agencies’ polls are.
We compared polls produced by major television networks with those produced
by Gallup and Rasmussen. We found that, taken as a whole, polls produced by
the networks were significantly to the left of those produced by Gallup and
Rasmussen.
We used the available data to provide a tentative ordering of the major television
networks’ polls from right to left. Our order (right to left) was: FOX, CNN, NBC
(which partners with the Wall Street Journal), ABC (which partners with the
Washington Post), CBS (which partners with the New York Times). These results
appear to comport well with the commonly held informal perceptions of the
political leanings of these agencies.
We also compared tracking polls produced by Gallup, Rasmussen, Hotline/FD,
and the Daily KOS. Here again we found significant evidence of bias. Most
notably, the Rasmussen and the Gallup polls were significantly to the right of the
Daily KOS poll