Tipping of the 2004 election
November 3, 2008 7:17 PM Subscribe
What exactly was it in 2004 that suggested in the final weeks that John Kerry was going to win? Case in point here and here (yeah, I know the latter is a Democrat site). Maybe I have selective memory but I thought a lot of pundits had expected a Kerry win. Or did many neutral sources have Bush pegged? If not, what I'm trying to figure out is where the forecast errors were made, and whether these same issues could produce a surprise tomorrow (not counting a "Bradley effect", since that was of course not a factor in 2004).
I'm not sure your premise is accurate.
I think Bush was leading going into election day on most polls.
http://www.realclearpolitics.com/bush_vs_kerry_hth.html
posted by bitdamaged at 7:37 PM on November 3, 2008
I think Bush was leading going into election day on most polls.
http://www.realclearpolitics.com/bush_vs_kerry_hth.html
posted by bitdamaged at 7:37 PM on November 3, 2008
Historically, polling has tended to overstate voting for Democrats and understate voting for Republicans. They create their samples based on assumptions about how many of each part will vote, and historically the pollsters have included more Democrats proportionally in their samples than actually voted on election day.
As a result, polls nearly always look better for the Democrats than the election result does.
What it was that suggested a Kerry win? It was people who believed what the polls said, despite the historical record of inaccuracy. That was, to some extent, confirmation bias: the polls were telling those commentators what they wanted to hear.
...which wasn't accidental. There was an incentive for the pollsters to do exactly that, to deliver a "product" to their customers which told those customers what they wanted to hear. Perhaps it was unconscious bias, but it seems likely that the pollsters jiggered their sampling percentages to fit their customers' preconceptions.
Only one major polling house called the 2004 election correctly, and it was working for Investor's Business Daily, a conservative magazine.
posted by Class Goat at 7:37 PM on November 3, 2008
As a result, polls nearly always look better for the Democrats than the election result does.
What it was that suggested a Kerry win? It was people who believed what the polls said, despite the historical record of inaccuracy. That was, to some extent, confirmation bias: the polls were telling those commentators what they wanted to hear.
...which wasn't accidental. There was an incentive for the pollsters to do exactly that, to deliver a "product" to their customers which told those customers what they wanted to hear. Perhaps it was unconscious bias, but it seems likely that the pollsters jiggered their sampling percentages to fit their customers' preconceptions.
Only one major polling house called the 2004 election correctly, and it was working for Investor's Business Daily, a conservative magazine.
posted by Class Goat at 7:37 PM on November 3, 2008
Well, that electoral-vote map shows the state of the polls. The difference between that map and the real result is that two of the three "barely Kerry" states went to Bush and that's 47 electoral votes. Honestly, they should not have been listed as "barely" anyone. They were clearly within the margin of error for a tie, and when you such big blocks of votes just hanging around at 50/50, nothing is certain.
You can see this in the 2nd link too:
posted by smackfu at 7:39 PM on November 3, 2008
You can see this in the 2nd link too:
The Harris Poll in three key states also affirmed the strong likelihood of a Kerry victory:2-4 point margins in polls that probably have 3-4% margins of error is not "strong" anything.
Florida - Kerry leads 51-47 percent of LV's
Pennsylvania - Kerry ahead 50-48 percent of LV's
Ohio - Kerry up 51-47 percent of LV's
posted by smackfu at 7:39 PM on November 3, 2008
A lot of people were reasoning like this:
1) The results in 2000 were so close that all the Democratic candidate would need to do to win would be to make even the tiniest improvement on them.
2) You hear a lot about people who supported Bush in 2000 switching to Kerry in 2004. It's hard to imagine anyone doing the opposite -- voting for Kerry in 2004 after voting for Bush in 2000.
3) From (2), it follows that Kerry will do better against Bush than Gore did against Bush in 2000.
4) From (1) and (3), it follows that Kerry will win.
posted by Jaltcoh at 7:41 PM on November 3, 2008 [2 favorites]
1) The results in 2000 were so close that all the Democratic candidate would need to do to win would be to make even the tiniest improvement on them.
2) You hear a lot about people who supported Bush in 2000 switching to Kerry in 2004. It's hard to imagine anyone doing the opposite -- voting for Kerry in 2004 after voting for Bush in 2000.
3) From (2), it follows that Kerry will do better against Bush than Gore did against Bush in 2000.
4) From (1) and (3), it follows that Kerry will win.
posted by Jaltcoh at 7:41 PM on November 3, 2008 [2 favorites]
Bush was ahead in the national polls, but Kerry was tied or narrowly ahead in FL and OH, which led some people to think he'd eke it out. There was also much talk that undecideds would break heavily against the incumbent. Combine that with the leaked early exits getting everyone's hopes up on election day and you have the popular memory that Kerry was ahead and lost because the polls were wildly off.
posted by shadow vector at 7:42 PM on November 3, 2008
posted by shadow vector at 7:42 PM on November 3, 2008
(Sorry -- in point #2, of course, I meant it's hard to imagine anyone voting for Bush in 2004 after voting for Gore in 2000.)
posted by Jaltcoh at 7:42 PM on November 3, 2008
posted by Jaltcoh at 7:42 PM on November 3, 2008
The exit polling for the day overstated Kerry's lead by about 5 or 6 points.
The McCain camp put out a little memo about it today, which while it has the intention of shoring up support, isn't too far off the mark in its description.
posted by wfrgms at 7:46 PM on November 3, 2008 [1 favorite]
The McCain camp put out a little memo about it today, which while it has the intention of shoring up support, isn't too far off the mark in its description.
posted by wfrgms at 7:46 PM on November 3, 2008 [1 favorite]
E-v.com has the 2004 election map progression starting here.
Kerry was leading for most of the summer and had 270 EVs as of Aug 29. The Swiftboating had begun in earnest earlier that month, resulting in a shift of 60 EVS to Bush.
Kerry recovered and as late as Oct 20 had 291 EVs, 298 on Nov 1.
The difference, in the end, was 68,242 votes in Ohio.
Animated map here.
posted by troy at 7:49 PM on November 3, 2008
Kerry was leading for most of the summer and had 270 EVs as of Aug 29. The Swiftboating had begun in earnest earlier that month, resulting in a shift of 60 EVS to Bush.
Kerry recovered and as late as Oct 20 had 291 EVs, 298 on Nov 1.
The difference, in the end, was 68,242 votes in Ohio.
Animated map here.
posted by troy at 7:49 PM on November 3, 2008
1. Harris has the largest house effect in favor of Democrats.
2. I think every meta-poll showed Bush with a lead
posted by milkrate at 8:31 PM on November 3, 2008
2. I think every meta-poll showed Bush with a lead
posted by milkrate at 8:31 PM on November 3, 2008
Don't forget that some 2.5 to 3 million ballots were thrown out in 2004 due to "irregularities". Whether that means there were hijinks at play (by *cough*Republicans*cough*) is unknown, but we do know that the vast majority of those trashed votes came from predominantly Democratic-leaning counties, especially in New Mexico and Ohio, among other places. I believe that Kerry, in fact, did win, but unfortunately that's now relegated to conspiracy theory.
posted by zardoz at 8:48 PM on November 3, 2008 [1 favorite]
posted by zardoz at 8:48 PM on November 3, 2008 [1 favorite]
If you look back a few days on Electoral Vote to October 30, you'll see that Bush had the lead that day, 280 to 243. On October 28, Kerry had the lead, 260 to 254.
The polls swung back and forth between Bush and Kerry. Some of the likely voter models of the polls ended up being flawed - Bush's ground game in 2004 was better than expected, and more Republicans turned up than expected. This resulted in as much as a two to three point difference between the polls and reality. The exit polls were up to 5 points off - in part because of a daytime-evening voter divide (Bush voters were more likely to vote after work, while Kerry voters were more likely to vote during the day ... exit polls were released before those after-work voters could all be interviewed), and in part because of the flawed assumptions of the likely voter models.
Your question doesn't ask this, but it leaves the implicit question "why won't this year be similar?" Two reasons: Massive early voting numbers with a 10 to 20 point Democratic lean (2 million people have already voted in Georgia, for instance, compared to 3.4 million in 2004 for the entire election cycle, including election day), and a tremendous ground game by Obama's campaign. Read the last 2 months worth of posts at fivethirtyeight for more information on this topic.
posted by Happydaz at 8:56 PM on November 3, 2008
The polls swung back and forth between Bush and Kerry. Some of the likely voter models of the polls ended up being flawed - Bush's ground game in 2004 was better than expected, and more Republicans turned up than expected. This resulted in as much as a two to three point difference between the polls and reality. The exit polls were up to 5 points off - in part because of a daytime-evening voter divide (Bush voters were more likely to vote after work, while Kerry voters were more likely to vote during the day ... exit polls were released before those after-work voters could all be interviewed), and in part because of the flawed assumptions of the likely voter models.
Your question doesn't ask this, but it leaves the implicit question "why won't this year be similar?" Two reasons: Massive early voting numbers with a 10 to 20 point Democratic lean (2 million people have already voted in Georgia, for instance, compared to 3.4 million in 2004 for the entire election cycle, including election day), and a tremendous ground game by Obama's campaign. Read the last 2 months worth of posts at fivethirtyeight for more information on this topic.
posted by Happydaz at 8:56 PM on November 3, 2008
One blogger noticed something odd about the polls: Obama has been polling better on weekends than on weekdays. The difference was a couple of points.
Why? Presumably because the people reached on weekends are statistically different than the ones reached on week days.
Which are more representative of the electoral result? Ah, that's the question.
posted by Class Goat at 9:08 PM on November 3, 2008
Why? Presumably because the people reached on weekends are statistically different than the ones reached on week days.
Which are more representative of the electoral result? Ah, that's the question.
posted by Class Goat at 9:08 PM on November 3, 2008
FWIW, electoral-vote.com on Nov 2, 2004 had the tally at 262 Kerry and 261 Bush, plus 15 exactly tied. That's pretty obviously a tie, statistically speaking--and situation where any number of factors could push the final result one way or the other.
On the other hand, even one day earlier Kerry looked like he had something of a lead--but not a strong lead by any stretch of the imagination.
posted by flug at 9:38 PM on November 3, 2008
On the other hand, even one day earlier Kerry looked like he had something of a lead--but not a strong lead by any stretch of the imagination.
posted by flug at 9:38 PM on November 3, 2008
Class Goat, this may or may not be the blog post you're thinking of.
The question does indeed remain if those week-end cellphone-type voters (presumably young people) actually get off their asses this time around. I have an endless lack of faith in my generation, but I think this might be the election we, you know, do something.
posted by McBearclaw at 11:14 PM on November 3, 2008
The question does indeed remain if those week-end cellphone-type voters (presumably young people) actually get off their asses this time around. I have an endless lack of faith in my generation, but I think this might be the election we, you know, do something.
posted by McBearclaw at 11:14 PM on November 3, 2008
I can speak to what the ground game looked like, because I was working for a GOTV 527 with overt Democratic sympathies. We canvassed my working-class union-heavy neighborhood for weeks and encountered livid hostility to Bush at every turn. There were a few holdouts (mostly of two implacable camps: "Taxes", and "Iraq"), but for the most part we were focusing on wishy-washy sometime-Democratic-voters who we felt needed an extra push to get to the polls, and they were generally determined and needed little suasion. During all this time I encountered other Kerry-related GOTV canvassers, but not one single Republican on the streets.
Election Day the GOP had a busy rides operation, though, something that I don't think the Democrats here were doing. You saw those cars everywhere in the neighborhood all day long. Was it all arranged by phone? I suppose it was. At the polls, the lines were incredible. Conventional wisdom was that more turnout = Democratic win, so we were all energized.
The final tally for my ward was for Kerry, but by a much thinner margin than we expected. The state (Wisconsin) went for Kerry, but again the margin was slim (less than a single percentage point). And as the returns came in they did look fairly good for the Democrats. There were all those Ohio issues, though. I think we imagined that it would all, again, come down to turnout. But it didn't.
I confess to extrapolating, and to assuming that because I didn't see anyone trying to canvass "my" people, there was no GOP ground game at all (where I was). But there was, and it really tapped into the silent majority stereotype. It was a learning experience.
posted by dhartung at 11:32 PM on November 3, 2008 [1 favorite]
Election Day the GOP had a busy rides operation, though, something that I don't think the Democrats here were doing. You saw those cars everywhere in the neighborhood all day long. Was it all arranged by phone? I suppose it was. At the polls, the lines were incredible. Conventional wisdom was that more turnout = Democratic win, so we were all energized.
The final tally for my ward was for Kerry, but by a much thinner margin than we expected. The state (Wisconsin) went for Kerry, but again the margin was slim (less than a single percentage point). And as the returns came in they did look fairly good for the Democrats. There were all those Ohio issues, though. I think we imagined that it would all, again, come down to turnout. But it didn't.
I confess to extrapolating, and to assuming that because I didn't see anyone trying to canvass "my" people, there was no GOP ground game at all (where I was). But there was, and it really tapped into the silent majority stereotype. It was a learning experience.
posted by dhartung at 11:32 PM on November 3, 2008 [1 favorite]
There was a major Zogby poll that came out right before the election:
"Late afternoon on Election Day, awfully late for a final call, Zogby predicted that Kerry would win Florida, Ohio, Iowa, and New Mexico (0 for 4!) and get at least 311 votes in the Electoral College, while Bush was assured of only 213."
See here
The link explains where Zogby's results were particularly atrocious, and suggests the reason was a dramatic underestimate of GOP voter turnout.
posted by lockedroomguy at 5:01 AM on November 4, 2008
"Late afternoon on Election Day, awfully late for a final call, Zogby predicted that Kerry would win Florida, Ohio, Iowa, and New Mexico (0 for 4!) and get at least 311 votes in the Electoral College, while Bush was assured of only 213."
See here
The link explains where Zogby's results were particularly atrocious, and suggests the reason was a dramatic underestimate of GOP voter turnout.
posted by lockedroomguy at 5:01 AM on November 4, 2008
...a dramatic underestimate of GOP voter turnout.
In essence, Zogby assumed going in that the Democrats were going to win, and then set up his stratified sample so that he got that result. The polling didn't really matter; the fix was in as soon as he determined how many Democrats and how many Republicans he was going to include in his sample.
posted by Class Goat at 10:34 AM on November 4, 2008
In essence, Zogby assumed going in that the Democrats were going to win, and then set up his stratified sample so that he got that result. The polling didn't really matter; the fix was in as soon as he determined how many Democrats and how many Republicans he was going to include in his sample.
posted by Class Goat at 10:34 AM on November 4, 2008
Class Goat is propogating yet another Republican lie.
The myth of the Liberal Mainstream Pollster is just as much a fabrication as the myth of the Liberal Mainstream Media. Note that Class Goat's example of a "Liberal-biased" pollster, Zogby, called the 2008 popular vote +1 McCain only days before the election, plus it consistently over-estimated Republican/Democrat turnout this year. Does this mean this "Liberal-biased" pollster became a "Conservative-biased" pollster in the last four years? No, it's just a shitty pollster. The only other useful fact Class Goat called out is that exit polling consistently over-estimates the Democrats (and this year Obama).
As we saw two days ago, aggregation sites like 538, Pollster.com (or, hey, that notorious pinko aggregator RealClearPolitics) nailed the election, and did so, not only because their analysis was good, but also the data (good analysis is useless if the data is systematically wrong). As someone else mentioned above, Pollster also called 2004 fairly accurately. The problem is that the election, just as in 2000, was a statistical tie in many key states.
The main problem here is a misunderstanding of statistics. If a pollster calls a state for Kerry +1% with a 3 or 4 percent margin of error (typical for an individual pollster), then it is just as likely that Bush wins by 1% as Kerry by they same amount. The problem here is sites like Electoral-Vote that are too aggressive calling a lean, not a biased pollster. Honestly, this is why I never paid much attention to Electoral-Vote. They only published the latest poll and did no real analysis.
The lessons learned from 2008?
1.) In the aggregate, the mainstream pollsters are pretty good at what they do. There are some quibbles, like cellphone only voting, and expected turnout, but they seem to more-or-less get things right.
2.) Pay far more attention to aggregation sites than individual pollsters, for the same reasons you pay attention to multiple independent studies rather than individual data in any science.
3.) Pay particular attention to fivethirtyeight.com. As someone at Wonkette said, Nate Silver taught numbers how to fuck. His simulation called every state but Indiana (and even the betting sites, historically better predictors of elections, didn't call Indiana until a few hours before polls closed, and I believe Silver himself said Indiana would flip) and maybe one elector in Nebraska. He also called the popular vote to within 1%. More importantly, he has had amazing articles covering all aspects of polls and how to interpret them. He knows his shit possibly better than anyone in the country.
4.) Pay no attention to exit polls (I don't think any of the networks did this year).
5.) If someone tells you that the mainstream pollsters got something wrong because they're all in the tank for Obama/McCain/Kerry/Bush/Kodos/Kang/Charles Nelson Reilly, ignore them. They have a vested interest in making you believe as such.
6.) Alaska is either way, way hard to poll, or way, way corrupt. I suspect probably both.
posted by dirigibleman at 9:52 PM on November 6, 2008
The myth of the Liberal Mainstream Pollster is just as much a fabrication as the myth of the Liberal Mainstream Media. Note that Class Goat's example of a "Liberal-biased" pollster, Zogby, called the 2008 popular vote +1 McCain only days before the election, plus it consistently over-estimated Republican/Democrat turnout this year. Does this mean this "Liberal-biased" pollster became a "Conservative-biased" pollster in the last four years? No, it's just a shitty pollster. The only other useful fact Class Goat called out is that exit polling consistently over-estimates the Democrats (and this year Obama).
As we saw two days ago, aggregation sites like 538, Pollster.com (or, hey, that notorious pinko aggregator RealClearPolitics) nailed the election, and did so, not only because their analysis was good, but also the data (good analysis is useless if the data is systematically wrong). As someone else mentioned above, Pollster also called 2004 fairly accurately. The problem is that the election, just as in 2000, was a statistical tie in many key states.
The main problem here is a misunderstanding of statistics. If a pollster calls a state for Kerry +1% with a 3 or 4 percent margin of error (typical for an individual pollster), then it is just as likely that Bush wins by 1% as Kerry by they same amount. The problem here is sites like Electoral-Vote that are too aggressive calling a lean, not a biased pollster. Honestly, this is why I never paid much attention to Electoral-Vote. They only published the latest poll and did no real analysis.
The lessons learned from 2008?
1.) In the aggregate, the mainstream pollsters are pretty good at what they do. There are some quibbles, like cellphone only voting, and expected turnout, but they seem to more-or-less get things right.
2.) Pay far more attention to aggregation sites than individual pollsters, for the same reasons you pay attention to multiple independent studies rather than individual data in any science.
3.) Pay particular attention to fivethirtyeight.com. As someone at Wonkette said, Nate Silver taught numbers how to fuck. His simulation called every state but Indiana (and even the betting sites, historically better predictors of elections, didn't call Indiana until a few hours before polls closed, and I believe Silver himself said Indiana would flip) and maybe one elector in Nebraska. He also called the popular vote to within 1%. More importantly, he has had amazing articles covering all aspects of polls and how to interpret them. He knows his shit possibly better than anyone in the country.
4.) Pay no attention to exit polls (I don't think any of the networks did this year).
5.) If someone tells you that the mainstream pollsters got something wrong because they're all in the tank for Obama/McCain/Kerry/Bush/Kodos/Kang/Charles Nelson Reilly, ignore them. They have a vested interest in making you believe as such.
6.) Alaska is either way, way hard to poll, or way, way corrupt. I suspect probably both.
posted by dirigibleman at 9:52 PM on November 6, 2008
« Older The site with the thing and the stuff? | What NYC hotel should I pick for a Christmastime... Newer »
This thread is closed to new comments.
posted by snookums at 7:35 PM on November 3, 2008 [1 favorite]