This article first appeared in the St. Louis Beacon: September 23, 2008 - The Pew Research Center has been the go-to source for empirical support that, with proper weighting, telephone political polls that do not include cell phone numbers still yield accurate estimates about candidate support, party preference, and political attitudes.
Now Pew is not quite so sure.
In a Sept. 23 release , the Pew Center reports on its three most recent national polls that included both land line households and cell-phone-only households. Interpreted separately, there continues to be no statistically significant difference between the standard land line sample and one that blends both land lines and cell phones. Here are the results:
- In a survey from June 18-29, the standard landline number were Obama 46 percent and McCain 41 percent; the blended land line/cell numbers were Obama 48 percent and McCain 40 percent.
- For a July 31-Aug. 10 survey, Obama and McCain were each at 45 percent on a standard land line survey; the blended land line/cell number showed Obama with a 46-43 percent lead.
- In the standard land line survey from Sept. 9-14, Obama and McCain were still at 45 percent, whereas the blended landline/cell numbers had Obama at 46 percent and McCain at 44 percent.
But the Pew team, headed by veteran survey expert Scott Keeter, notes that on each of these three surveys Obama does one to two percentage points better and McCain receives one or two points less when cell phone results are incorporated.
In previous surveys, there were no differences between land line responses and cell phone answers after controlling for age. Cell-phone-only citizens within each age segment had the same distribution pattern as did land-line respondents. That is still the case for voters 30 and over but, on the last two polls, it was not so for those under 30. For voters ages 18 to 29, the cell phone only group is 62 percent Obama/27 percent McCain while the land line segment is 52 percent Obama/39 percent McCain.
Pew concludes that “even though the omission of cell phones from election polls does not currently make a large difference in the substantive results, (our) surveys this year suggest at least the possibility of a small bias in landline surveys” and that “such a bias could be consequential in an election that appears to be very competitive right now.”
The Obama supporters at my neighborhood coffee house Sunday were subdued. They were reading the Post-Dispatch/KMOV poll on the presidential race in Missouri, trying to accept the results: Their hero trailed John McCain by 4 percentage points. How could that be?
Then one of them struck a consoling note. Maybe the poll was wrong. The Research 2000 firm conducting the survey only sampled households with land line telephones. That meant that anyone relying solely on cell phones — now more than 14 percent of the adult population — was systematically excluded.
More to the point, the cell-phone-only segment is disproportionately under 35, prime territory for Obama voters especially among the college-educated young adults. About one out of four among this group regard land lines as so 20th century. If some of them had been polled, would not that probably alter the results, turning the McCain lead into something close to dead heat?
Sound reasoning — wrong conclusion. The premises are indeed accurate. Cell-phone-only percentages are rising, and they are concentrated among younger adults. But land-line young voters do not have significantly different political attitudes and preferences than do their cell-phone-only counterparts. So, if a pollster weights the sample by age and education so that each segment is represented in the final results proportional to its share in the overall electorate, the estimates remain unbiased.
The empirical buttress for this approach is a series of national surveys conducted regularly since 2004 by the Pew Research Center (www.pewresearch.org ) using separate land line and cell phone samples. Its latest poll, fielded in late June, reinforces the continuing finding that “comparisons between land-line samples and combined land-line and cell samples have found little or no difference in overall results.”
But why do survey researchers rest on this fortuitous finding? There is no assurance that land-line and cell-only respondents, weighted demographically, will continue to think similarly about matters political. Why not include the cell segment just to be more confident? For surveys covering the entire United States, that is indeed happening. The Gallup Poll’s daily tracking surveys as well as the New York Times/CBS polls, for example, now sample both land lines and cell phone numbers. Interviewing cell-phone respondents costs about twice as much as questioning land-line holders but these organizations are willing to pay the price.
But incorporating cell phones for state surveys is logistically impossible. Unlike land lines, a cell phone’s area code and exchange do not link it directly to a specific geographic location. A pollster can easily purchase a land line telephone sample for, say, Missouri. The vendors know the state’s area codes, exchanges and working banks for the final four digits. They then drop the last two digits, substitute another two numbers generated randomly, and — voila — you have a sampling frame where every land line has an equal chance to be selected, an essential condition for using the probability theory that generates those margin of error (e.g., plus-or-minus 3.8 percent) statements. Conversely, a cell-phone number might have a 314 or 636 area code, for example, but its user could be living anywhere.
At least one major pollster, John Zogby, argues for avoiding cell phones. In addition to citing the Pew findings and the cost implications, he makes the point that people answer the land line at home while a cell phone call can come in a wide range of settings. Writing in the September 2008 Politics, the leading trade magazine for campaign consultants, he points out that “if you call someone at a crowded bus stop, they might not answer candidly or they may find themselves in loud surroundings and be unable to concentrate properly.”
Nevertheless, continuing to assume that the one-out-of-four young adults with cell phones will continue to vote like the three-out-of-four 18 to 34 years olds with land lines carries risks, especially in an election in which that group could make the difference for Barack Obama.
Terry Jones is a polling expert and professor at the University of Missouri at St. Louis.