In light of recent attempts by Republicans to soften the blow of polling showing a solid Obama lead (attempts which have been widely mocked), I highly recommend this very detailed analysis by Mark Blumenthal from three months ago on how polling methodologies differ, and why these differences are so crucial this election year:
As a Pew Research Center study recently demonstrated, random-sample surveys continue to provide accurate data on most measures — but only when their samples of telephone numbers include both landline and mobile phones, and only when the completed interviews are weighted to match the demographic composition of the population. That means the weighting procedures that pollsters use are critical to producing accurate results.
The need to weight accurately by race and ancestry is particularly significant when it comes to evaluating the contest between Obama and Romney. As Gallup itself reported in early May, Romney led Obama among non-Hispanic white voters by 54 to 37 percent, while the president had the support of more than three-quarters of non-white registered voters (77 percent). Obama’s support among African Americans on Gallup’s tracking poll stood at 90 percent.
That gap makes the way pollsters account for race hugely important. When pollsters weight their samples to match population demographics, every percentage point increase in black representation translates into a nearly one-point improvement in Obama’s margin against Romney. The difference of just a few percentage points in the non-white composition of a poll can produce a significant skew in its horse race results.
Interestingly, many analysts — Andrew Sullivan, perhaps most notably — are very skeptical of Rasmussen’s polls, which consistently show much better numbers for Romney. Aaron Blake at The Fix has more on the firm’s historical record:
Rasmussen has had both good years and bad years, according to various pollster ratings. While its track record was pretty good in the middle of last decade (2004 and 2006) and average in 2008, after the 2010 election the New York Times’ Nate Silver labeled Rasmussen “biased and inaccurate.” Silver calculated that Rasmussen missed the final margin of the races it polled in the 2010 midterms by an average of 5.8 percentage points.
But Republicans note that Rasmussen did just fine in the last presidential race in 2008. They also note that Gallup, while its top-line number is different from Rasmussen, has shown similar movement in its daily tracking poll in recent days.
“Rasmussen’s track record (’08 and ’10) makes it a very credible polling source in this year’s election,” Romney pollster Neil Newhouse told The Fix in an e-mail.
If there continues to be a disparity between Rasmussen and other polls, expect to hear plenty more about Rasmussen’s numbers — along with the continuing debate about how reliable they are.
- Most Accurate Pollster in 2008: Rasmussen (riehlworldview.com)
- Skewed and Unskewed Polls (pjmedia.com)
- Conservative Site: Mitt Romney Is Winning, If You Just Change the Polls to Make Him Win [Polls] (gawker.com)
- There’s A New Polling Site That Conservatives Are Excited About – Here’s Why You Should Totally Ignore It (businessinsider.com)
- Conservatives Embrace Alternate Polling Reality (buzzfeed.com)
- Making Their Own Reality, And How! (sporkinthedrawer.typepad.com)
- Are The Polls Biased Against Mitt Romney? (outsidethebeltway.com)
- The Website Where Mitt Romney’s Winning in a Landslide (alternet.org)
- Mitt Romney would lead eight in unskewed data from newest CNN/ORC poll (tarpon.wordpress.com)
- Polls Are Now Part Of The Liberal Conspiracy, Ctd (andrewsullivan.thedailybeast.com)