How Useless are Polls?

Sefarst

Graduate Poster
S
Joined
Feb 27, 2007
Messages
1,237
So far in this election, we've seen the polls and the pundits make spectacular failures when predicting the direction voters will take. Case in point, the Newsweek poll from 5 days ago and today's latest Gallup poll.

Newsweek said:
The latest NEWSWEEK Poll shows the Democrat with a 15-point lead over McCain.
Click to expand...

I saw this headline and, forgetting that these polls are garbage, thought to myself, it looks like McCain can kiss this one goodbye.

Today, however, Gallup gives us a different result:

Gallup said:
The latest Gallup Poll Daily Tracking update on the presidential election finds John McCain and Barack Obama exactly tied at 45% among registered voters nationwide.
Click to expand...

So a difference of 15% in only 5 days? Am I missing something? If not, why do they waste their time taking these polls and then waste my time featuring them on the news?
 
Well it's way early for polls anyway. Have they even had a debate yet? For that matter, are they even officially the nominees from each party?
 
All polls are subject to variation, and anybody who's worked with random number generators knows that the variations can be quite wide. If you flip a coin 100 times and it comes up heads 60, have you really discovered a flaw in probability or just a random variation?

That said, there are many ways to influence a poll result. Polling on weekends is said to skew poll results towards the Democrats, because Republicans are more likely to be away from home. Polling registered voters (as in the LA Times/Bloomberg poll today) as compared to "likely" voters also tends to skew things towards the Democrats.

More important, no pollster these days uses their raw data. They look for demographics and other indicators in the polling data they gathered and try to adjust for oversampling in particular groups. When a pollster says they polled 1000 people and 48.1% picked Obama, that does not mean 481 picked Obama. It means that after they adjusted the number, 481 was their estimate. Note: I am not opposed to polling organizations doing this, just reporting the fact that they do.

What I do recommend, though, is that consumers of polls pay attention to which ones get it right and which ones get it wrong. At least early in the primary season, I noted that SurveyUSA was doing great at predicting contests accurately, while Mason-Dixon and Reuters/CSpan/Zogby were doing very poorly.

But at this point in the process they are more used the way a drunk uses a lamppost; for support, not illumination.
 
forget it...

Plummy beat me to the joke.
 
Last edited:
I know I saw, I saw.

To the OP:

Polls are interesting, but shouldn't be taken too seriously just yet.

We need some debates.
 
I would say the thing that has to be taken into consideration is that not only is there the built in margin of error due to sample size, there's also error due to the fact that all polls have to deal with the fact that respondents are somewhat self-selected and also that respondents can lie. But I feel that if you can take lots and lots of polls, take into account their historical behavior relative to actual elections, and consider both previous trends of the data and try to extrapolate how public opinions may change in the future, you can crank more meaningfulness out of them than you would a single poll.

There is a fairly substantial part of me which feels that the government (perhaps on the state level, but certainly something that can use the threat of the libertarian's ever popular "men with guns") should be able to force people to participate in polls in a mechanism similar to jury duty. As long as polls are optional, you are going to get biased results. (Although the actual polling would have to be done by independent pollsters that would contract with the government so as to avoid corruption and to make people not feel that their answers are being monitored by the government.)
 
UserGoogol said:
There is a fairly substantial part of me which feels that the government (perhaps on the state level, but certainly something that can use the threat of the libertarian's ever popular "men with guns") should be able to force people to participate in polls in a mechanism similar to jury duty. As long as polls are optional, you are going to get biased results. (Although the actual polling would have to be done by independent pollsters that would contract with the government so as to avoid corruption and to make people not feel that their answers are being monitored by the government.)
Click to expand...
But if the government forced everyone to participate in polls, wouldn't that skew the results even more given that many people are not registered voters or even likely voters? At least with the system now we can weed out the "no interest" part of the population.

Are you for mandatory voting?
 
There are several factors that can cause a poll result to be off, the first simply being randomness. A poll of about 1000 people is said to have a margin of error of about 3.5% (with about 95% confidence), but that assumes also that the sample is indeed representative. If the sample is not truly representative for some reason, which can be hard to pin down, that could add an extra bias besides randomness.

So even if the methodology is perfect, a poll can be more wrong than the margin of error (about 1 in 20 times), and it is not likely that the methodology is perfect (i.e. that the sample is truly representative of those who will actually vote).

I think the RCP average of polls is probably better than any one poll, and probably Gallup and Rasmussen are better single polls because of larger samples. Gallup is the most famous poll.
 
Puppycow said:
There are several factors that can cause a poll result to be off, the first simply being randomness. A poll of about 1000 people is said to have a margin of error of about 3.5% (with about 95% confidence), but that assumes also that the sample is indeed representative. If the sample is not truly representative for some reason, which can be hard to pin down, that could add an extra bias besides randomness.

So even if the methodology is perfect, a poll can be more wrong than the margin of error (about 1 in 20 times), and it is not likely that the methodology is perfect (i.e. that the sample is truly representative of those who will actually vote).

I think the RCP average of polls is probably better than any one poll, and probably Gallup and Rasmussen are better single polls because of larger samples. Gallup is the most famous poll.
Click to expand...



...and sometimes they poll for the answers they want in the Head-line.



:irule:13:
 
You must log in or register to reply here.

Back
Top Bottom