Are phone polls biased toward landlines?

September 8, 2008 12:42 PM

How accurate are political polls in the age of the cell phone? Here's why I ask: Out of nine folks in my office, seven of us have no landlines in our homes; we are mobile exclusive. Of the those seven, six are voting Obama and one is voting McCain. The two of us with landlines are split one to one for Obama-McCain. Any pollsters working in our neck of the woods would see the race in a dead heat, one to one. In reality, we're seven to two. So, aren't the polls growing more skewed with every passing year? Do some pollsters actually call cell-phone users? If not, there's a bias.
posted by jackypaper to society & culture (10 answers total)

Previously.
posted by enn at 12:45 PM on September 8, 2008

The excellent FiveThirtyEight.com addressed this very issue. Basically, the damage from this is currently minimized because the populations most likely to be cell-only (young people, minorities...) are also less likely to vote in general, and it's possible to compensate - at least in part - by increasing the weight of of those populations' results to compensate for their unrealistically low appearance in your sample. For now, it probably isn't biasing polls too much. As time goes on, it's going to presumably exceed that point that can be easily compensated for by things like weighting, and polls really will start to have problems.
posted by Tomorrowful at 12:48 PM on September 8, 2008

From the link I posted in the previous AskMe, which was from the methodology section of a survey of New Orleans residents three years after Katrina:

An address-based sample with a mixed-mode interviewing design was used to ensure that households without telephones, those with unlisted telephone numbers, and those that are cell-phone only were included. As such, 669 interviews were completed by telephone, 447 were in-person interviews, and 178 were self-administered via the Internet.

Polling firms are working around this. The additional methods are expensive and time-consuming, but they are being incorporated, and weighting methods are being changed/developed to account for cell phone-only demographics.
posted by rtha at 12:52 PM on September 8, 2008

I've read that polling firms are now dialing completely randomized 10 digit numbers to get cell phone users.
posted by prozach1576 at 1:29 PM on September 8, 2008

prozach, they cannot do such a thing, I'm pretty sure. It's against regs. And running the math, there's 10 million phone numbers per area code. There's something like 200 area codes, which means no possible way they're going to get very far by calling random numbers.
posted by Lemurrhea at 2:13 PM on September 8, 2008

200 area codes * 10 million per = 2 billion numbers. Assume for the purposes of this very rough calculation that there's one phone number for each of 300 million Americans. That means that approximately 15% of the numbers generated by random-digit dialing will be valid.

Which is why people do, in fact, use random-digit dialing.
posted by ewiar at 2:36 PM on September 8, 2008

Note that cell phones are blocked from being robo-called, but telephone surveys per se are not illegal. Pollsters are also not blocked by the Do Not Call Registry.
posted by dhartung at 2:37 PM on September 8, 2008

I worked for a polling firm a few years back, though not in the US.

Cell phone only households where getting to be a problem for the general surveys. Although at the time weighing the data wasn´t that much of a problem, but it was pretty obviusly a problem that was an is getting bigger for the polling firms.

I don't know the US regs and what options the pollsters have for optaining cell numbers. But we tried to buy some cell numbers from one provider and test them a few times, with limmited success.

I'd assume American pollsters can buy sets of cell numbers just like we could. If not from the providers then from third party data collectors.

They generally contain a LOT of duds. Call screening was a lot more persuasive and we got a lot of discontinued 'pay as you go' numbers etc.

On the other hand, when you get through to a cell, you generally get 10-15 percentage point higher answer rates, since you avoid the "Stop" questions especially in election polls ie "could i please talk to the person on the house hold that last celebrated his/her birthday" or the US equivalent.

But all thats probably only of interest for the polling firms in election surveys, since its pretty costly and surveying is all about cost cutting. I expect they will keep on use mostly landlines in anything that can´t be verified like ellection results.
posted by Greald at 3:49 PM on September 8, 2008

I was once called on my cellphone by a major polling agency. I missed the call so I'm not sure what it was about, but when I returned the call the answer message let me know who it was. So unless my experience is totally bizarre (I wish I had answered!) someone out there is calling cell phones.
posted by Solon and Thanks at 4:04 PM on September 8, 2008

Yeah, they are definitely calling cell phones. It's just more expensive (because cell phone numbers must be dialed by a human, not an auto-dialer) and time-consuming.
posted by rtha at 7:12 PM on September 8, 2008