How Accurate are the 2008 Polls?

Wednesday, October 22. 2008

How Accurate are the 2008 Polls?

D. J. Drummond writes, "The polls are wrong this year, very wrong." (via Instapundit)

I believe that Obama is leading McCain right now, but, like others, I question the purported margin of the lead.

For supporting evidence, examine the pollsters' most recent, past performance.

In 2006, the Democrats won the aggregate US House vote by 7.9% over the Republicans.

Yet, an average of 35 different 2006 polls compiled from October through the November 2006 election purportedly showed the Dems with a 14.2% point lead over the Repubs in the generic House vote -- an overestimation of more than 6% points.

It's certainly something to keep in mind, as we wrote back in November 2006.

Here's the list:

FOX News 11/04 - 11/05, Democrats +13
CNN 11/03 - 11/05, Democrats +20
USA Today/Gallup 11/02 - 11/05, Democrats +7
ABC News/Wash Post 11/01 - 11/04, Democrats +6
Pew Research 11/01 - 11/04, Democrats +4
Newsweek 11/02 - 11/03, Democrats +16
Time 11/01 - 11/03, Democrats +15
CBS News/NY Times 10/27 - 10/31, Democrats +18
NBC/WSJ 10/28 - 10/30, Democrats +15
CNN 10/27 - 10/29, Democrats +11
Cook/RT Strategies 10/26 - 10/29, Democrats +23
Newsweek 10/26 - 10/27, Democrats +15
FOX News 10/24 - 10/25, Democrats +11
AP-Ipsos 10/20 - 10/25, Democrats +19
Zogby 10/20 - 10/23, Democrats +11
Hotline/FD 10/19 - 10/23, Democrats +18
USA Today/Gallup 10/20 - 10/22, Democrats +13
CNN 10/20 - 10/22, Democrats +17
ABC News/Wash Post 10/19 - 10/22, Democrats +14
Cook/RT Strategies 10/19 - 10/22, Democrats +22
Pew Research 10/17 - 10/22, Democrats +11
Newsweek 10/19 - 10/20, Democrats +18
NBC/WSJ 10/13 - 10/16, Democrats +15
CNN 10/13 - 10/15, Democrats +16
FOX News 10/10 - 10/11, Democrats +9
Harris 10/06 - 10/09, Democrats +12
USA Today/Gallup 10/06 - 10/08, Democrats +23
CNN 10/06 - 10/08, Democrats +21
CBS News/NY Times 10/05 - 10/08, Democrats +14
ABC News/Wash Post 10/05 - 10/08, Democrats +13
Cook/RT Strategies 10/05 - 10/08, Democrats +11
Newsweek 10/05 - 10/06, Democrats +12
Time 10/03 - 10/04, Democrats +15
AP-Ipsos 10/02 - 10/04, Democrats +10

Posted by Tom Elia in Polls at 10:15 | Comments (37) | Trackbacks (0)

Trackbacks

Trackback specific URI for this entry

No Trackbacks

Comments

Display comments as (Linear | Threaded)

I don't think the polls are wrong so much as people responding are a self selecting group. There are a number of different reasons for this but beyond simple over sampling somewhere there's a factor of what the demographics are of people who feel like responding and I think that trends to the left.

#1 Bandit on 2008-10-22 13:08 (Reply)

Could be, I don't know.

All I know is, pollsters in general missed pretty badly last election cycle -- I think precisely because their models describing party identification were off by substantial margins...

#1.1 Tom Elia on 2008-10-22 13:12 (Reply)

Comparing generic house polls to Presidential polls isn't a very good idea.

#2 Dan on 2008-10-22 13:45 (Reply)

Why so? Particularly if party id is the driving force behind the poll...

#2.1 Tom Elia on 2008-10-22 14:10 (Reply)

The best analysis of all publicly released polls this year is at http://www.fivethirtyeight.com - this is the guy who was on Colbert a week ago or so. He's a baseball stats geek who veered into political poll analysis.

#3 Twan Digler (Homepage) on 2008-10-22 13:48 (Reply)

...o it could be due to voting fraud on behalf of the Republicans.

#4 Brian on 2008-10-22 13:49 (Reply)

Yes indeed. You've 'nailed it'...

#4.1 Tom Elia on 2008-10-22 14:08 (Reply)

Problem with this analysis:

You're attempting to compare presidential polls to congressional polls. Worse, you're comparing national trend on congressional polls to national trend polls on Presidential politics.

Traceable accuraccy on congressional polling would have to follow polling on a race-by-race basis, as it's well known that everybody hates Congress but thinks their congressman is a local hometown hero. In terms of congress, congress is literally hundreds of different people. Presidential polls are generally about the same two people nation wide.

Also, local state polling is more accurate in presidential politics.

All in all, you're attempting to compare apples and bananas-- these two things don't even have the same shape.

#5 Anonymous on 2008-10-22 13:51 (Reply)

I'm not sure I agree completely with you -- the congressional generic numbers generally follow the party identification numbers, which is why I think there is some validity in the comparison.

#5.1 Tom Elia on 2008-10-22 14:07 (Reply)

How do the 2004 Presidential Election polls compare to the actual 2004 Presidential Election outcome?

I think that would paint a more valid picture of the accuracy of the current Presidential Election polls.

#5.1.1 Anonymous on 2008-10-22 14:15 (Reply)

You make a very good point, one with which I would normally agree.

However, the similarity between 2006 and 2008 is the party id assumptions made in the polls vary greatly from those made in 2000 and 2004.

And, as I point out, these predictions missed by a fair amount -- I would argue due to the party id assumptions.

#5.1.1.1 Tom Elia on 2008-10-22 14:29 (Reply)

Pure inanity. This post says nothing -- it barely implies some kind of hope to not get crushed.

30 Years of rationalized selfishness starting with Reagan are coming to a close.

#6 Anonymous on 2008-10-22 13:52 (Reply)

Excellent analysis. You've completely refuted the argument in one cogent argument.

Thanks for stopping.

#6.1 Tom Elia on 2008-10-22 14:08 (Reply)

So what you're telling me is...

USA Today/Gallup
ABC News/Wash Post
Pew Research

nailed it just before the election?

#7 massoluk on 2008-10-22 14:12 (Reply)

They were definitely the best, yes.

Though I would note that they all seemed to change drastically at the end of the election cycle. Perhaps because they adjusted their party id?

USA Today/Gallup showed the Dems +23 in a poll from 10/06-08, and +13 on another taken 10/22-25.

Gallup ended at Dems +7 -- a 16-point move in about a month.

ABC News/Washington Post was at +13 in a poll taken from 10/05-08; and then +14 in another taken 10/19-22.

The ABC poll ended up at +6 -- moving 8 points in around 2 weeks.

Pew was at +11 in a poll taken 10/17-22.

The Pew poll ended up +4, moving 7 points from there in about 2 weeks as well.

I wonder what accounted for those moves?

#7.1 Tom Elia on 2008-10-22 14:25 (Reply)

This is a ridiculous apples-to-oranges comparison. Poll figures for "generic Congressional vote" are in no way a model of expected aggregate voting across all races. No one casts a "generic" vote--votes are always cast for particular candidates.

#8 Ben on 2008-10-22 14:25 (Reply)

But that's how the question is posed -- and it relates to party id....

#8.1 Tom Elia on 2008-10-22 14:26 (Reply)

Yes, that's how the question is posed, and yes, it relates to party identification, but that doesn't make it a valid comparison. Citizens have reactions to particular candidates that may run counter to their party ID. Take Alaska: the GOP would assuredly win a majority in a "generic Senate candidate" poll but if Stevens is convicted he won't fare as well as the generic GOP candidate in the poll.

If there were a bunch of 'generic presidential candidate' polls then this would be a valid way of questioning the polls' reliability. But the polls aren't asking that question!

They still may be suspect... but this is a specious way of illustrating why they aren't to be trusted.

#8.1.1 Ben on 2008-10-22 16:37 (Reply)

I completely disagree! Especially when we see the disparities we saw in 2006.

In some cases, we were (are) seeing predictions of a larger party id disparity than we saw in 1932, 1964, or 1974...

That's a serious limb to climb out on...

#8.1.1.1 Tom Elia on 2008-10-22 16:40 (Reply)

It really doesn't matter to me how far off the polls are. What matters to me is that the polls seem to be unanimously in the tank for Democracy. It seems to me that they're saying if you want a Republic, or any variation thereof, go to China.

#9 Jouda Mann on 2008-10-22 14:35 (Reply)

You're not allowing for lost and flipped electronic votes.

Seriously, analysis of Ohio per precinct vote tallies made for some very very unlikely (say, 1 in 10,000,000) outcomes.

In almost ALL instances of recorded vote/loss/flip, the loss/flip favored a republican candidate/conservative initiative.

#10 Xboxershorts on 2008-10-22 15:02 (Reply)

To which analysis are you referring?

#10.1 Tom Elia on 2008-10-22 15:05 (Reply)

There's a lot of analysis on this. The best compendium is probably th excellent book, "Was the 2004 presidential election stolen?" While the book contains some unnecessary partisan crap, the actual presentation and analysis of the data is wholly sound. And wholly startling.

#10.1.1 Ben on 2008-10-22 16:21 (Reply)

Everyone knows that only lazy, freeloading dems are home during the day to answer these poll calls. So of course they are going to skew to the guy that wants to keep handing out freebies.

#11 Pete T. on 2008-10-22 15:07 (Reply)

Polls also don't take into account the busloads of people traveling from Illinois to vote in Ohio. No one fixes an election like a Chicagoan!

#12 ssssmashing on 2008-10-22 15:12 (Reply)

To Tom Elia

Read John Conyer's book, and then use google to find and read Ohio lawyer Bob Fitrakis' well documented legal proceedings.

The fact you have no knowledge of this shows how poorly the media has approached this shameful legacy.

Electronic voting is a joke and a sham.

#13 Xboxershorts on 2008-10-22 15:18 (Reply)

Actually, I am familiar with the allegations -- I just wanted to see what you referenced.

Conyers and Fitrakis are both extreme partisans. Perhaps you could point to other sources documenting the same claims?

I'm no fan of electronic voting, per se, but I would point out that Ohio has gone to a non-incumbent Dem just once in the last 50 years (heck FDR lost Ohio in 1944!).

#13.1 Tom Elia on 2008-10-22 15:43 (Reply)

While the book contains some unnecessary partisan crap

You shoulda stopped right there

#13.2 Bandit on 2008-10-23 12:29 (Reply)

Obama will win the electoral college by a comfortable margin, over 300 electoral votes. This will be the first clear majority for a Democrat since Carter won in 76. That says nothing about the democratic candidate, only that the republican party is as damaged by ineptitude, corruption, short-sighted policies and the bad economy as it was then.

Republicans will adopt a poisoned well strategy and declare that the election was stolen by those outrageous people who actually registered and voted (but they'll call it voter fraud). They will spend the next 4 years brooding and recriminating and will push another neocon fundamentalist wacko in 2012 and fail again. By 2016, the party will implode and something else will take its place.

#14 Ted on 2008-10-22 15:22 (Reply)

That's probably true. That's what happened to the democrats in 2000 and 2004 and now they've been replaced by the marxist party.

#14.1 Anonymous on 2008-10-22 15:45 (Reply)

Actually, if Obama wins a majority of the popular vote, he will be only the fifth Dem since the Civil War to have done so.

If I may, I would like to ask a question: How does one become a neocon fundamentalist?

#14.2 Tom Elia on 2008-10-22 15:47 (Reply)

did you know that 60% of statistics are made up?

#15 i love stats on 2008-10-22 15:34 (Reply)

But that's only 12% of the time.

#15.1 Tom Elia on 2008-10-22 15:48 (Reply)

Incorrect Tom,

Thor Hearne is an example of an extreme partisan. J kenneth Blackwell is an extreme partisan.

Attack the messenger without challenging the message? A common tactic of non denial.

Are you afraid of the data?

#16 Xboxershorts on 2008-10-22 15:56 (Reply)

Conyers and Fitrakis aren't partisans? But Blackwell is?

How do you figure?

Conyers, who has been in the House since being elected in 1964, ran pretend impeachment hearings last year, and Fitrakis ran for Ohio gov. on the Green Party ticket.

Here's my denial denial: Repub voter fraud did not swing Ohio in 2004.

How's that?

#16.1 Tom Elia on 2008-10-22 16:01 (Reply)

DONT FORGET MARGIN OF ERROR

Saying that polls were off by 6% really is not that bad given that each candidate probably has a +/- of 3/4 points.

#17 Greg on 2008-10-22 16:55 (Reply)

Something like 19 of these 35 polls had the Dems' margin of victory at +14 or better (or were outside a margin of error of 3%) ...

#17.1 Tom Elia on 2008-10-22 17:02 (Reply)

Add Comment

Name
Email
Homepage
In reply to
Comment	Enclosing asterisks marks text as bold (word), underscore are made via _word_. Standard emoticons like :-) and ;-) are converted to images. E-Mail addresses will not be displayed and will only be used for E-Mail notifications
	Remember Information? Subscribe to this entry

Syndicate This Blog

Major Newspapers,
Magazines,
and Web Sites

Political Web Sites,
Magazines, and
Other Journals

Research Tools

Blogs

Regional Newspapers

Features

Economics, Science, & Technology Related
Blogs and Web Sites

Categories

Business News

Libertarian Ad Network

Old Archives

Wednesday, October 22. 2008

How Accurate are the 2008 Polls?

Search

Supporter

Syndicate This Blog

Major Newspapers,Magazines,and Web Sites

Political Web Sites,Magazines, andOther Journals

Research Tools

Blogs

Regional Newspapers

Features

Economics, Science, & Technology RelatedBlogs and Web Sites

Categories

Business News

Libertarian Ad Network

Old Archives

Wednesday, October 22. 2008

How Accurate are the 2008 Polls?

Search

Supporter

Major Newspapers,
Magazines,
and Web Sites

Political Web Sites,
Magazines, and
Other Journals

Economics, Science, & Technology Related
Blogs and Web Sites