I'm fixing a poll where my campaign wins and stops my base from wandering.

November 3, 2012 5:05 PM

What's the difference between internal and public polls?

Throughout the election season I keep hearing about the internal polls campaigns do and how they can differ from all the public polls that get reported. What is the real difference between the two? I've often heard that they can skew to the benefit of whoever commissioned the poll, but wouldn't the campaigns have an interest in getting the most accurate polling possible? Are they using different sampling methods or asking different questions? What's the distinction?

posted by fishmasta to law & government (13 answers total)

I think you have to divide the public rhetoric surrounding internal polls from the reality. Internals are just premier polls: more expensive, better call centers, better analysis, calling cellphones too, etc. They are, in fact, better, because as you say campaigns want to have actually good information to base their actions on. One of the criticisms of Nate Silver is that he had access to elite Barack Obama campaigns internals, and that's why he was able to get so close to the real result.

What campaigns say about their internals is a totally different thing. Losing campaigns always claim to have internals that show they're really winning. That doesn't make it so.
posted by gerryblog at 5:25 PM on November 3, 2012

The simplest definition is that an internal poll is paid for by candidate and a public poll is paid for by a news organization. The details and exact questions are published for a public poll and generally not for an internal poll.

An internal poll can have several purposes. In some cases they will be trying out various talking points to see which are most effective, ror example to figure out if the voters are more concerned about economic issues or social issues.

Other internal polls are called push polls -- propaganda disguised as a poll. For example they will have a question such as: "Would the fact that [opponent] has been arrested for spousal abuse cause you to be more likely (press 1) or less likely (press 2) to vote for him.

Internal polls can also be used to figure out where they are more likely to get donations.

Internal polls can also be used to generate the illusion of "momentum" which is important for keeping donations flowing, so biases are often desired. Generally you will hear about internal polls that are good for the candidate but not ones that look bad for the candidate. They may use the unbiased data for internal use but released the biased data to the news.
posted by JackFlash at 5:27 PM on November 3, 2012 [3 favorites]

There are two main differences: One, organizations that have an incentive based on the poll results can run polls (especially push-polls, polls that have a certain view they advocate for) that help maintain that narrative. They can have separate polls for internal strategy use.

The second difference is that internal polls often aren't set up the same way as general polls: they may make assumptions about turnout or the electorate that aren't necessarily true (a lot of assumptions in polling are more art than science) or may be primarily internally used for polling of a single (or circumscribed) demographic; e.g. single women. Using that poll to then extrapolate to the general state of an election introduces noise. The poll is still useful to the campaign because it does things like test messaging and retention of messaging, but isn't a reliable indicator of the broader state of the election.
posted by klangklangston at 5:28 PM on November 3, 2012

One of the criticisms of Nate Silver is that he had access to elite Barack Obama campaigns internals, and that's why he was able to get so close to the real result.

There is no evidence that this is true. In fact the the origin of stuff like this is on Breitbart.com and Redstate.com. There is no reason to believe that internal polls are "elite" or any better than public polls.
posted by JackFlash at 5:36 PM on November 3, 2012 [6 favorites]

there is no reason to believe that internal polls are "elite" or any better than public polls.

That's just simply not true.

On the one hand, you have public polls that survey 500 (or 1000, or sometimes on rare occasions as many as 10,000) registered and/or likely voters.

On the other hand, you have a campaign that has made over 100 million calls and doorknocks. And that's not counting paid ID's or robos.

Do a little power analysis on that one and then tell me again that internals aren't more elite than public polls.
posted by dersins at 6:42 PM on November 3, 2012

I've worked with pollsters on internal polls quite a bit in my work.

The biggest thing about internal polls is that they tend to be used to find out very specific information that the campaign needs, like "how are we polling with suburban female voters?" or "how does this particular message do with persuadable voters?" Whereas public polls for elections tend to be more about the head-to-head numbers (or at least, that's what gets reported on, regardless of what else the pollsters report in the crosstabs).

I think another thing is that the "consumers" of each types of polls are different. Public polls are really targeted at journalists and bloggers, who then write stories/blog posts about them. So public pollsters who are wrong once in a while don't have a lot of accountability, and it can actually be sort of good, because they'll get a lot of attention for being an outlier. Only pollsters that are consistently wrong face any sort of accountability.

The audience for internal polls is campaign staffers, who really need polls to be accurate, because they use the polls to make resource decisions. If a pollster makes errors that lead a campaign to misuse resources, the campaign leadership is not going to be happy, and they're likely to remember that for the next campaign. So there's more incentive to be accurate.

To clear something up: push polls or voter ID calls are not internal polls. Those are a different beast - the former is a communications tactic and the latter is what field organizers and volunteers do to identify supporters and undecided voters.
posted by lunasol at 6:45 PM on November 3, 2012 [4 favorites]

The difference is who pays for them, and who pays for them determines what gets asked and how the results are disseminated.

I work for an organization that has commissioned polls on our campaigns and issues. Sometimes we're testing language, or targeting specific audiences, and we're always trying to get good information. We may or may not make the results public, or we may make some of them public and not others, depending on what we're trying to do. We are often very careful with how things are asked. We're not trying to get different information from a public poll, such as one done by a newspaper or TV station, but we might, depending on how the question is asked.

And what lunasol said -- voter ID calls are very definitely not polls, and are working with a different universe of people to call.
posted by gingerbeer at 7:09 PM on November 3, 2012 [1 favorite]

On the other hand, you have a campaign that has made over 100 million calls and doorknocks. And that's not counting paid ID's or robos. Do a little power analysis on that one and then tell me again that internals aren't more elite than public polls.

Look, there are two issues here.
1. There is absolutely no evidence that the Obama campaign gave their internal polls to Nate Silver. The claims come from crazy right wing sites that are convinced that Obama is Muslim communist. This "elite" poll stuff is just black helicopter paranoia.

2. 100 million calls and doorknocks has nothing to do with scientific polling. That's just a get-out-the-vote effort. That information would be useless to Silver because it isn't an unbiased sample.
posted by JackFlash at 7:15 PM on November 3, 2012 [9 favorites]

To clear something up: push polls or voter ID calls are not internal polls. Those are a different beast - the former is a communications tactic and the latter is what field organizers and volunteers do to identify supporters and undecided voters."

You're totally right when it comes to what campaigns think of as internal polls. However, plenty of campaigns send out the "results" of push polls or voter ID calls, which are then sometimes described in media as "internal polls."
posted by klangklangston at 8:39 PM on November 3, 2012

Not trying to push the derail further, but I was of the impression (and I read Nate Silver religiously and black helicopter sites only when tricked) that he had had access to Obama internal polls in 2008, but had no access and was not using them now, as it violated the NY Times' independence rules, etc. Not that a quick Google can dig it up now.

In any case, internal polls don't have any "elite" methodology that public pollsters don't have access to. But polling costs money -- good polling even more so; in particular, calling cell phones requires humans dialing numbers, which is incredibly expensive but avoids bias given the huge shift to cell phone only households in the modern era. Sample size is also subject to diminishing marginal returns; with a large population and 95% CI, you need 600 surveys to be +/-4%, but 2400 to be +/- 2% and over 9600 to be +/- 1%.

It's hard to imagine that any Presidential campaign -- but especially the 2008 Obama campaign, famously both the wealthiest campaign and the most data-driven campaign in the history of the US presidency -- wouldn't outspend the (financially dying) media on polling. In fact, they spent $28 million on polling and research, more than they spent on Internet media, and 10 times what McCain spent.
posted by Homeboy Trouble at 9:02 PM on November 3, 2012

This is a fascinating discussion between Mark Halperin and Charlie Rose on polls and campaign strategy. It seems like the primary purpose of the internal polls is to determine campaign strategies. How to attack an opponent. Should Obama run on his own record and accomplishments or just attack Romney as too extreme? Or, attack Romney as a flip-flopper? Should Romney move to the center, or hold consistent positions? Should Romney be specific about the details of his tax plan, or not? etc. I wonder how effective this polling is; it certainly seems to be bad for the country.

At 13:10 Halperin says (paraphrasing), "One of the poll numbers that has turned around pretty dramatically is: you ask people, 'regardless of who will you vote for, who do you think is going to win?' And clearly Governor Romney has made progress both in favorability rating and in 'who do you think is going to win?' More people still think the President's going to win, but depending on which poll you look at, the gap has narrowed dramatically. You must have that be relatively narrow in order to win."

And, Haplerin says this again on Morning Joe, "Romney needs to go into the weekend, for his own sake, with the race tied in the conventional wisdom rather than what some people are now thinking, that these polls suggest the President will win this with the Electoral College dominance that he’s had."

I think this is revealing. If both parties believe that conventional wisdom must be that their candidate is close or ahead in order for him or her to win, then they obviously have a motivation to create the perception that their candidate is close, regardless of what they believe the truth to be. And, they are probably right, imo. What the public polls say now has a big impact on how/if people will vote. Thus, anything the Democrats or Republicans report publicly about their internal polls is highly suspect. It is not surprising that Halperin reports both that he now believes the election is either close or Romney will win big, and that "Republicans now have a chance to win the Senate." Maybe he truly believes this, but there is good reason to think he is exaggerating these claims in order to impact public perception. There is good reason to be suspicious about individual public polls as well, imo.

Halperin claims the difference between the polls all comes down to "undecided voters" and "independents." Are undecideds going to vote or not? Are "independents" going to vote for Romney or Obama? Republicans claim previous poll data shows that independents tend to vote against incumbents, and many undecideds are unlikely to vote, and that many of the public polls aren't reflecting this accurately.

but wouldn't the campaigns have an interest in getting the most accurate polling possible?

Yes, but it is also in their best interest for any public polling information to reflect that their candidate is going to win.
posted by Golden Eternity at 9:48 PM on November 3, 2012

Yeah, and it's often worth remembering that polling contractors will answer any question that you want asked, pretty much, so you can get internal polls that answer bad questions in the affirmative. That combines with campaigns where there's internal divisions on strategy and that turns into "leaks" of the polling that supports the strategy of one faction, and you have another grain of salt to take internal polls with. Most internal polls stay internal; there's usually an incentive at play when they don't.
posted by klangklangston at 11:45 PM on November 3, 2012 [1 favorite]

There is no evidence that this is true. In fact the the origin of stuff like this is on Breitbart.com and Redstate.com.

No, it appears the origin is Slate author Sasha Issenberg's The Victory Lab, p. 290 (the wording is very similar to the gloss on Buzzfeed). Neither source, however, suggests that his having access to those internals was a way to improve his own analysis -- in fact, the motivation came from the Obama campaign wanting to check those internals against his analysis.

Silver reported on the 2010 midterm elections as a Times employee/contractor and was fairly accurate compared with other prognosticators.

100 million calls and doorknocks has nothing to do with scientific polling.

Well, you can match it up with other statistical data such as household income and make some analytical use of it. There's a strong response bias caveat, though. I experienced that this year when a state assembly campaign used its voter contacts to reinforce a belief that their candidate was the frontrunner; in fact, he placed third. I did tell them about my concerns before releasing the information.
posted by dhartung at 5:09 PM on November 4, 2012