Rasmussen
2/4/10; 500 likely voters, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Colorado
2010 Governor (trends)
49% Hickenlooper (D), 45% McInnis (R)
Favorable / Unfavorable
Scott McInnis: 52 / 29
John Hickenlooper: 56 / 36
Sarah Dutton breaks down polling on Sarah Palin pre-Tea Party Convention.
Del Ali defends his survey of Republican voters for DailyKos.
Carl Bialik examines how errors permeated Census micro-data.
Survey Practice releases their February issue.
Democracy Corps (D) / Common Cause / Change Congress / Public Campaign Action Fund
2/2-4/10; 805 likely voters, 3.5% margin of error
Mode: Live telephone interviews
(Democracy Corps: toplines, Summary)
National
State of the Country
35% Right Direction, 58% Wrong Track (chart)
Obama Job Approval
47% Approve, 47% Disapprove (chart)
Favorable / Unfavorable
Republican Party: 33 / 43
Democratic Party: 38 / 44
Barack Obama: 49 / 40 (chart)
2010 Congress: National Ballot
46% Democratic candidate, 45% Republican candidate (chart)
Rasmussen
2/3/10; 500 likely voters, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Nevada
2010 Governor (trends)
44% R. Reid (D), 35% Gibbons (R)
45% Sandoval (R), 33% R. Reid (D)
40% R. Reid (D), 36% Montandon (R)
Favorable / Unfavorable
Jim Gibbons: 35 / 63 (chart)
Mike Montandon: 39 / 29
Brian Sandoval: 53 / 30
Rory Reid: 40 / 52
Marist
2/1-3/10; 1,072 adults, 3% margin of error
910 registered voters, 3.5% margin of error
(all questions asked of registered voters unless otherwise marked)
Mode: Live telephone interviews
(Marist release)
National
Obama Job Approval
44% Approve, 47% Disapprove (chart)
Dems: 81 / 10 (chart)
Reps: 15 / 80 (chart)
Inds: 29 / 57 (chart)
Favorable / Unfavorable
Barack Obama: 50 / 44 (chart)
State of the Country (asked of all adults)
38% Right Direction, 54% Wrong Track (chart)
2012 President
44% Obama (D), 29% Palin (R), 15% Bloomberg (i)
If the 2010 election for congress were held today, would you support your current congressperson who represents your district in Washington D.C. or would you vote for someone else?
42% Current Congressperson, 44% Someone else
Rasmussen
2/5-6/10; 500 likely voters, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Ohio
2010 Senate
Portman (R) 43%, Fisher (D) 39% (chart)
Portman (R) 42%, Brunner (D) 38% (chart)
Favorable / Unfavorable
Rob Portman: 45 / 24
Lee Fisher: 38 / 39
Jennifer Brunner: 43 / 32
Job Approval / Disapproval
Pres. Obama: 49 / 51 (chart)
Gov. Strickland: 46 / 53 (chart)
My column this week looks at the controversy over a series of surveys conducted by SurveyUSA for the liberal web site Firedloglake. Please click through to read the whole thing.
Lost in the attack memos and other questions raised is an important question facing nearly every telephone survey conducted in House, Senate and Gubernatorial races this year: Are we at the point where the majority of true "likely voters" under the age of 35 are out of reach of landline telephone samples? And at what point is simply "weighting up" those younger voters that pollsters can still reach inadequate to solve the problem?
The table below, produced by the Pew Research Center and based on their national surveys, shows that by 2006 their unweighted landline samples were under-representing roughly a third of adults under age 35. And that was as of three years ago, when the percentage of all adults living in landline-only households was estimated at 12%, nine percentage points lower than the most recent estimate:
Now consider the estimated growth in the cell-phone-only population over the last three years. As shown in the chart below (which comes from a report last year by the National Center for Health Statistics of the Centers for Disease Control and Prevention), landline-only samples are most likely to miss voters under age 35.
Now consider this additional statistic reported on Pollster.com by Mike Mokrzycki in December. On the most recent CDC report covering the first half of 2009, nearly two thirds (63.5%) of people age 25-29 live in households with either no landline phone (45.8%) or in "cell-mostly" households (17.7%), those were "all or almost all calls are received on cell phones."
So what should a pollster do if they reach so few 18-to-34-year-old voters that they make up just 1% of the likely voters sample for an election where past turnout suggests that age group should make up roughly 10% of the electorate? If the pollster believes they have under-represented younger voters, can they simply weight to correct the problem? Not if the shortfall is that extreme. In a sample with only 400 or 500 completed interviews, such a weight would multiply 4 or 5 interviews by a factor of 10. As I wrote in the column, you don't need to be a statistician to imagine how those "super respondents" might crate greater error and volatility in the results, especially those produced by cross-tabulations of demographic subgroups.
Let's remember that we are able to pick at SurveyUSA because they were willing to disclose the weighted demographics of their sample and because they opted against any such extreme weighting in this case. So rather than beat up on SurveyUSA, we might do better to ask: How many polls have we seen in recent months that involved a similarly sparse number of younger likely voters and were simply weighted up by factors of 5 or greater to conceal the shortfall? How would we know?
Finally, whatever we want to make of the Firedoglake surveys, it is important to remember that SurveyUSA has maintained an outstanding record of final-poll accuracy, especially in U.S. House elections and in hard-to-model primary elections. For House races, the company's own scorecard -- which I have no reason to doubt -- shows that their average error on the margin in polling 27 House races in 2006 (3.4) was roughly half that of all other pollsters combined (6.3). Their error rate was also significantly lower than the three most prolific public pollsters that year, Research2000 (5.5), Zogby (5.9) and RT Strategies (5.9).
So since we have picked at their work mercilessly, I want to give SurveyUSA's Jay Leve the last word and reproduce the full email he sent me last week in response to my questions about the Firedoglake surveys:
In August 2002, SurveyUSA released a poll showing US Senator Robert Torricelli (D-NJ) trailing. No survey to that point had showed Torricelli trailing. An hour after the poll was released, SurveyUSA's client, CBS-TV in Philadelphia, called SurveyUSA and said, "Put your helmets on. The DSCC is coming after you." And the DSCC did. The DSCC found a journalist willing to write the smack that the DSCC was shoveling, and the message went forth: Nothing wrong with Robert Torricelli, plenty wrong with SurveyUSA.
A few weeks later, Torricelli dropped out of the race. Other polls had the same results as SurveyUSA.
Fast forward to today: In a poll conducted in January 2010, at a time the Democrats were losing the state of Massachusetts, SurveyUSA finds an incumbent Democrat in a tight fight in New York state. The DCCC is unhappy. Partisans start shoveling smack. "Sources" start providing willing journalists with leaked memos. Nothing wrong with Democrat Tim Bishop. Plenty wrong with SurveyUSA.
The highway to high office is littered with the road kill of political operatives who find it easier to campaign against a poll than an opponent.
Lost in the hurly burly is an opportunity for real reflection. To my knowledge, there has never (ever) been a publicly released telephone poll conducted in a U.S. congressional district that included a known subset of interviews with respondents who did not have a home (aka: landline) telephone. An acknowledged limitation of SurveyUSA's work in NY-01, and a known limitation to date of all congressional district polling, is that voters who do not have a home phone are under represented. At a statewide-level (in contrast to the CD level), only one pollster in the 2009 election included a known subset of cellphone-only respondents in its sample (at extraordinary expense, because of the theoretical justification), and that pollster's results were worse than many polling firms who did not include a known subset of cell-phone-only respondents. Whether one anticipates that in 2010 young voters will turn out in record numbers of stay home in record numbers, the problem of how to count those voters is real, and right before us.
Ron Brownstein reviews the demographics of Congressional Districts (don't miss the interactive map).
Gallup releases data showing 36% of Americans have a positive view of socialism and adds Obama job approval to their "State of the States" feature.
Matthew Yglesias thinks ideological self-reports are not very useful.
Frank Newport takes questions on the public's state of mind, the Tea Party Convention, and how elected officials can benefit from polling.
Joshua Tucker adds his caveats to the Daily Kos Republicans poll.
Chris Bowers finds little support for cutting government.
Jonathan Bernstein considers the limits of polling on health reform (via Sullivan).
National Journal's insiders project big losses for Democrats in 2010.
Andrew Gelman shares more back-and-forth with David Runciman.
PPP sees a partisan divide in Super Bowl allegiances.
Marist finds 49% think issue ads aren't appropriate for the Super Bowl.
Zogby says 59% of Americans plan to watch the Super Bowl.
Economist / YouGov
1/31-2/2/10; 1,000 adults, 3.7% margin of error
Mode: Internet
(Economist release)
National
Obama Job Approval
46% Approve, 48% Disapprove (chart)
Dems: 75 / 20 (chart)
Reps: 12 / 86 (chart)
Inds: 39 / 58 (chart)
Economy: 41 / 53 (chart)
Health Care: 40 / 54 (chart)
Congressional Job Approval
10% Approve, 67% Disapprove (chart)
2010 Congress: Generic Ballot
43% Democrat, 38% Republican (chart)
State of the Country
32% Right Direction, 53% Wrong Track (chart)
Overall, given what you know about them, do you support or oppose the proposed changes to the health care system being developed by Congress and the Obama Administration?
46% Support, 54% Oppose (chart)
Rasmussen
2/2/10; 500 likely voters, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Nevada
2010 Senate (trends)
45% Lowden, 39% Reid (chart)
47% Tarkanian, 39% Reid (chart)
44% Angle, 40% Reid
44% Krolicki, 41% Reid
Favorable / Unfavorable
Harry Reid: 44 / 55 (chart)
Sue Lowden: 48 / 27
Danny Tarkanian: 50 / 35
Sharron Angle: 37 / 30
Brian Krolicki: 40 / 33
Job Approval / Disapproval
Pres. Obama: 46 / 54 (chart)
Gov. Gibbons: 39 / 59 (chart)
DailyKos.com (D) / Research 2000
2/1-3/10; 600 likely voters, 4% margin of error
Mode: Live telephone interviews
(Kos release)
New Hampshire
2010 Senate: Republican Primary
36% Ayotte, 27% Lamontagne, 4% Binnie
2010 Senate: General Election (trends)
46% Ayotte, 39% Hodes (chart)
46% Hodes, 36% Lamontagne
45% Hodes, 35% Binnie
2010 Governor: General Election (trends)
59% Lynch, 13% Kimball
Favorable / Unfavorable
Paul Hodes: 47 / 29
Kelly Ayotte: 54 / 24
Ovide Lamontagne: 34 / 39
William Binnie: 31 / 30
Barack Obama: 55 / 38 (chart)
Rasmussen
2/2/10; 500 likely voters, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Colorado
2010 Senate (trends)
Norton 45%, Romanoff 38%
Norton 51%, Bennet 37%
Wiens 42%, Romanoff 40%
Wiens 45%, Bennet 40%
Buck 45%, Romanoff 39%
Buck 45%, Bennet 41% (chart)
Favorable / Unfavorable
Ken Buck: 43 / 26
Michael Bennet: 42 / 40 (chart)
Andrew Romanoff: 40 / 37
Tom Wiens: 35 / 30
Jane Norton: 49 / 31
Job Approval / Disapproval
Pres. Obama: 45 / 53 (chart)
Gov. Ritter: 40 / 56 (chart)
Rasmussen
2/1/10; 500 likely votrs, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Connecticut
2010 Governor (trends)
40% Lamont, 37% Foley
37% Malloy, 36% Foley
41% Lamont, 33% Fedele
36% Malloy, 35% Fedele
Favorable / Unfavorable
Michael Fedele: 39 / 22
Tom Foley: 42 / 22
Ned Lamont: 43 / 35
Dan Malloy: 43 / 29
DailyKos.com (D) / Research 2000
2/1-4/10; 2,400 adults, 2% margin of error
Mode: Live telephone interviews
(Kos release)
National
Favorable / Unfavorable
Barack Obama: 56 / 42 (chart)
Nancy Pelosi: 40 / 51
Harry Reid: 26 / 64
Mitch McConnell: 20 / 62
John Boehner: 20 / 62
Democratic Party: 39 / 56
Republican Party: 32 / 59
State of the Country
39% Right Direction, 60% Wrong Track (chart)
McLaughlin & Associates (R)
1/13-14/10; 600 likely voters, 4% margin of error
Mode: Live telephone interviews
(McLaughlin release)
Florida
2010 Governor
41% McCollum (R), 30% Sink (D) (chart)
US: Palin, Tea Parties (CNN 1/22-24)
CNN / Opinion Research Corporation
1/22-24/10; 1,009 adults, 3% margin of error
Mode: Live telephone interviews
(CNN release)
National
Favorable / Unfavorable
Sarah Palin: 43 / 46 (chart)
Tea Party Movement: 33 / 26
Fox New / Opinion Dynamics
2/2-3/10; 900 registered voters, 3% margin of error
Mode: Live telephone interviews
(Fox release)
Update: Tea Party and 2010 Elections
National
Obama Job Approval
46% Approve, 47% Disapprove (chart)
Dems: 81 / 12 (chart)
Reps: 14 / 82 (chart)
Inds: 45 / 47 (chart)
Congressional Job Approval
22% Approve, 69% Disapprove (chart)
Favorable / Unfavorable
Barack Obama: 51 / 43 (chart)
George W. Bush: 38 / 55
Rahm Emanuel: 14 / 24
Nancy Pelosi: 24 / 52
Democratic Party: 42 / 48
Republican Party: 42 / 46
Party ID
36% Democrat, 36% Republican, 22% independent (chart)
Another update, this one on the volunteer exit poll conducted this week in Cook County Illinois by the recently launched Chicago Current. Current editor Geoff Dougherty posted a refreshingly candid postmortem on their efforts:
At 6:11 p.m. yesterday, before the polls closed, I wrote that our exit polling suggested Toni Preckwinkle had the Cook County Board president's race locked down.
And I was right. Our survey honed in on Preckwinkle's strong performance early in the day, and continued to highlight her lead as the election progressed.
And yet ... our poll was wrong. I predicted Preckwinkle would snag 69% of the vote, and noted that the poll had an 8% margin of error. Preckwinkle ended the day with 49% of the vote -- well outside that margin.
Such are the joys and pains of exit polling.
There's more, and it's worth clicking through to read the rest.
I would give the Current an "A" for effort and transparency, but we need to be realistic about the quality of the survey they ultimately produced. Dougherty says it cost just $200, "most of which went for a $100 rental car," and don't think he would argue with the conclusion that they got what they paid for. The poll managed to collect just 93 completed interviews at only 9 of 25 precincts (presumably) selected at random. As Dougherty reported at 1:32 p.m. on Tuesday:
So far we've got about 30 responses. We'll be taking a pause here as our field crew relocates to new spots and starts talking to voters.
We'd originally planned to survey 25 precincts, but logistics are interfering, and we'll probably wind up with about half that. We'd targeted 600 voters, but low turnout will probably leave us with about half of that count.
Never mind the very small sample size. How truly random was the sample? It's hard to tell from this description, but the execution clearly fell short of ideal.
Dougherty says that the "networks often pay tens of thousands of dollars for these things." That's not quite right. I'm not sure how it translates into a per-state cost, but the every-two-year National Election Pool (NEP) exit polling operation has a multi-million dollar budget (Voter News Services, VNS, the forerunner to NEP, operated in 2000 on a budget of over $35 million; my understanding is that current costs are much lower but still in the millions). Note that in most states of interest, NEP will sample 20 to 50 precincts. As the scale of what the Current was attempting in a single county was in line with the exit poll that NEP conducts in each state.
I write this post not to beat up on the Current -- again, I give them credit for enterprise and transparency -- but to remind my media colleagues that all "exit polls" are not created equal. Not by a long shot.
Update: The cost statistic I cited for VNS from 2000 is accurate but potentially misleading. VNS was responsible for both exit polls and reporting final vote counts for every race (the latter function is now provided by the Associated Press). The costs also vary considerably between presidential and off-year elections. Finally, the NEP exit operation still includes more than just exit polls, it also collects vote results at samples of key precincts and provides statistical modeling and analysis used to "call" races.
Gallup updates their "State of the States" interactive feature (via Lymari Morales).
David Hill says the GOP resurgence is more about candidate images than a sudden surge in brand appeal.
Chris Weigant points to a leveling off of Obama's approval in January.
Matt Continetti thinks avoiding a fall in Obama's approval ratings might not save Democrats in November.
Jim Geraghty finds the results of the Daily Kos Republicans poll not "that surprising or even that troubling."
Keith Olberman catches something ironic in the Frank Luntz memo on how to defeat financial reform (via Atlantic wire).
Andrew Gelman challenges David Runciman's views on public opinion toward health reform.
Gary Langer shares polling data from a Taliban stronghold in Afghanistan.
Pew finds that 93% of 18-29 year olds and 75% of 12-17 year olds own cell phones (via Susannah Fox).
Today we have yet another odd epilogue to story of Strategic Vision, LLC. Apparently not satisfied with their history of setting the low bar for basic disclosure about the surveys they claim to have conducted since 2004, the company is now attempting something new: Attempting to retroactively withdraw previous disclosure.
Until a few weeks ago, the content published at the company's web site, strategicvision.biz, had been automatically archived by the non-profit Internet Archive along with hundreds of thousands of other web pages. In my December column, I linked to two such pages (displaying polls conducted during 2005 and 2007**). As of today, however, if you search the Internet Archive for strategicvision.biz or try either of the links I used previously (and be forewarned: their heavily trafficked site is notoriously slow), you will encounter this error message:
Robots.txt Query Exclusion.
We're sorry, access to http://www.strategicvision.biz has been blocked by the site owner via robots.txt.
What that means is that sometime in January, someone at Strategic Vision added some code ("User-agent: ia_archiver Disallow: /") to a file on their web site that specifically blocks the Internet Archive from searching and displaying pages from their company web site. Let's be clear that Strategic Vision is well within its rights in blocking such searches, and has done nothing illegal or particularly nefarious. As explained on their Frequently Asked Questions (FAQ) page, the Internet Archive "is not interested in preserving or offering access to Web sites or other Internet documents of persons who do not want their materials in the collection," and thus provide instructions on how to "exclude any historical pages."
That said, given the swirl of accusations about Strategic Vision arising from a failure to disclose basic information about their methods, this new effort to scrub previously disclosed information from what is essentially a public library for the Internet is more than a little creepy. Combined their apparent blocking of access to strategicvision.biz to me and my colleagues at the National Journal, and we get a story of a company that keeps digging a deeper and deeper hole for itself.
By the way, all credit for spotting this latest twist in the story goes to Michael Weissman, the retired University of Illinois physics professor who previously published a "Fourier analysis" of Strategic Vision's results on FiveThirtyEight.com. His son Jonathan realized that Strategic Vision might delete their archive, and thus downloaded everything he could before it disappeared. So the archived pages live on -- undoing previous disclosure is harder than it looks.
**As of this writing we were still able to load some of the 2005 page (sporadically), and if you experience as similar result it is probably because of something gone awry at archive.org. The code in the robots.txt file on the Strategic Vision site shows that they want Internet Archive to remove stop displaying their content.
Rasmussen
2/2/10; 500 likely voters, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Kentucky
2010 Senate
49% Grayson (R), 35% Mongiardo (D)
48% Paul (R), 37% Mongiardo (D)
44% Grayson (R), 40% Conway (D)
47% Paul (R), 39% Conway (D)
Favorable / Unfavorable
Trey Grayson: 61 / 18
Dan Mongiardo: 45 / 43
Rand Paul: 54 / 26
Jack Conway: 47 / 32
Job Approval / Disapproval
Pres. Obama: 42 / 57
Gov. Beshear: 49 / 48
SurveyUSA / KWCH-TV / KCTV-TV
1/29-31/10; 519 likely Republican primary voters, 4.4% margin of error
Mode: Automated phone
(SurveyUSA release)
Kansas
2010 Senate: Republican Primary
40% Jerry Moran, 33% Todd Tiahrt
Rasmussen
2/3/10; 500 likely voters, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Illinois
2010 Senate
46% Kirk (R), 40% Giannoulias (D)
Favorable / Unfavorable
Mark Kirk: 55 / 33
Alexi Giannoulias: 46 / 39
Job approval / Disapproval
Pres. Obama: 54 / 45
Gov. Quinn: 45 / 53
Rasmussen
2/1/10; 500 likely voters, 4.5% margin of error
Mode: Automated phone
(Rasmussen release)
Connecticut
2010 Senate (trends)
54% Blumenthal (D), 35% Simmons (R)
56% Blumenthal (D), 36% McMahon (R)
Favorable / Unfavorable
Richard Blumenthal: 70 / 27
Rob Simmons: 60 / 26
Linda McMahon: 51 / 34
Job Approval / Disapproval
Pres. Obama: 51 / 49 (chart)
Gov. Rell: 67 / 33 (chart)
Ipsos / McClatchy
1/28-31/10; 1,127 adults, 3% margin of error
Mode: Live telephone interviews
(Ipsos release)
National
State of the Country
37% Right Direction, 57% Wrong Track (chart)
Obama Job Approval
50% Approve, 46% Disapprove (chart)
Dens: 79 / 19 (chart)
Inds: 53 / 31 (chart)
Reps: 19 / 79 (chart)
Congressional Job Approval
21% Approve, 74% Disapprove (chart)
As of right now, do you favor or oppose the healthcare reform proposals presently being discussed?
37% Favor, 51% Oppose (chart)
Party ID
30% Democrat, 26% Republican, 45% independent (chart)
I want to add a few thoughts to Emily's post earlier today on the DailyKos/Research 2000 poll of Republicans and how they might check for a skew in the sample that some argue would result from "sane Republicans" hanging up after taking offense to the questions. Another potential problem, called out today by Republican pollster Alex Lundry, is not as easy to check: The possibility of a skew in respondents' answers caused by what pollsters call "acquiescence bias."
Acquiescence bias is the tendency of some respondents to select affirmative answers where the choice is whether to affirm or reject the statement presented (including "agree or disagree," "favor or oppose" and "yes or no" formats). This topic has been the subject of decades of study and debate among social scientists, and even though pollsters continue to rely on agree-disagree questions, academic survey researchers mostly agree that this format tends to produce more apparent agreement than questions offer a choice between two competing statements.
Here is an example from Schuman and Presser's classic text, Questions and Answers in Surveys (p. 221), based on an experiment first conducted by the NORC General Social Survey in 1974: They asked a random half sample to agree or disagree with this statement: "Most men are better suited emotionally for politics than women." Slightly less than half (47.0%) agreed, 53.0% disagreed.
They asked the other random half-sample to choose between two statements (and included a middle choice):
Would you say that most men are better suited emotionally for politics than are most women, that men and women are equally suited, or that women are better suited than men in this area?
Fewer (33.1%) agreed that men were better, 4.3% said women were better suited than men, and 62.6% said they were both equally suited. Researchers at the University of Michigan's Survey Research Center replicated the experiment three times between 1974 and 1976, producing similar results. They produced consistently greater agreement that "men are better" using the agree/disagree format (ranging from 44.3% to 45.5%) than when using forced-choice format (ranging from 32.5% to 38.3%).
Another strategy to reduce this bias is to try to balance the direction of the statements, as recommended in Presser, et. al, Methods for Testing and Evaluating Survey Questions (p. 440):
Acquiescence bias can be reduced by balancing scales so that the affirming response half the time is in the direction of the construct and half the time is in the opposite direction (e.g. six agree/disagree items on national pride, with the patriotic response matching three agree and three disagree responses).
With those recommendations in mind, consider the questions asked on the DailyKos/Research2000 survey in the order in which they presented the results. The first eight present all of the more sensational, ludicrous assertions (most of which pertain to President Obama). Seven of eight ask respondents to affirm or reject the extreme statement:
They then ask 15 issue questions that do mix up the order somewhat. Eight questions -- ask respondents if they agree with a liberal policy position, five ask about a conservative policy position, and two (the questions about Christ and marriage as a partnership) force choices between two statements:
I don't want to overstate the consensus of pollsters -- academic or otherwise -- on this issue. Many highly regarded survey researchers continue to rely on agree/disagree questions, often because of their simplicity and brevity or because such questions are part of a long-standing time series that the pollster would rather not disrupt (good example of the latter here; for more discussion see Javeline, 1999).
So while it would be a bit unfair to condemn Research 2000 for relying on question formats that pollsters and academics continue to rely on, Lundry has a point. Acquiescence bias probably exaggerates the amount of agreement measured for some of the more ludicrous assertions about Barack Obama tested on the Kos poll.
Update: As Alex Lundry notes below, his comments about acquiescence bias earlier today came after reading a message sent by Stanford graduate student Josh Pasek to AAPOR's members only listserv. With Josh's permission, here is a portion of that message:
Given that 10-20% of respondents tend agree with any statement (likely due to social norms), I went through the survey mentally subtracting 15 percentage points from every "yes" answer. That does leave some shocking numbers -- particularly as acquiescence tended to indicate support for gay rights, sex education, etc. -- but suggests that Birthers, for instance, may be outnumbered in the party (a slight consolation at best). I'm not saying this to suggest that the opinions being expressed even with a correction are reasonable, but I worry that not addressing this kind of issue is the reason so many people out there are skeptical of survey results in the first place.
George Bishop and David Moore issue their 2nd annual Dubious Polling Awards.
Alan Abramowitz challenges the SurveyUSA/Firedoglake polls; Jay Leve responds (via Smith).
Mark Mellman says live-interviewer polls show a more plausible campaign narrative than IVR polls.
Nate Silver sees little demographic variation among Republicans in their perceptions of politics.
Charles Lemos questions the demographics of the Daily Kos Republicans poll.
Frank Newport takes a look at the popularity of recent Obama proposals.
Gallup lists the most conservative and liberal states.
Ezra Klein considers the role of legislative "process" in shaping public opinion on health care reform (via Chait), John Sides sees a different explanation.
Reid Wilson finds evidence of benchmark polling by typically safe-seat Democrats.
Democracy Corps digs deeper into dial testing of the State of the Union address.
Tom Jensen sees warning signs for Democrats in the Illinois turnout.
Harry Enten estimates where the close count in the Illinois GOP governor's race is going.
Trent Alexander, Michael Davern, and Betsey Stevenson discover problems in public use Census data stemming from efforts to project respondent confidentiality (via Sides); Justin Wolfers summarizes; Andrew Gelman and Doug Rivers react.
Magellan Strategies for Mark Kirk (R)
2/2/10; 885 likely voters, 3.3% margin of error
Mode: Automated phone
(Magellan memo)
Illinois
2010 Senate
Kirk 47%, Giannoulias 35%
Favorable / Unfavorable
Mark Kirk: 31 / 26
Alexi Giannoulias: 24 / 39
Barack Obama: 51 / 45
Marist
1/25-27/10; 838 registered voters, 3.5% margin of error
360 Democrats, 5.5% margin of error
Mode: Live telephone interviews
(Marist release)
New York
2010 Governor: Democratic Primary
23% Paterson, 70% Cuomo (chart)
2010 Governor: General Election
46% Lazio, 43% Paterson (chart)
64% Cuomo, 27% Lazio (chart)
Job Rating
Gov. Paterson: 26% Excellent/Good, 70% Fair/Poor (chart)
Pres. Obama: 46% Excellent/Good, 54% Fair/Poor (chart)
Yesterday, Daily Kos released a poll of self-identified Republicans which showed surprisingly high agreement on a series of questions asking whether Barack Obama should be impeached, is a socialist, is a racist, and if he was born in the United States, among other things. The poll has created much debate in the blogosphere about the extremism of Republicans in the U.S.
While the results of the survey may be troubling, some have asked whether they are really representative of all Republicans or only the most extreme. Newsweek's Katie Connolly speculates, for example, that the potentially loaded questions that follow two more conventional probes of vote likelihood and 2012 vote preference may have caused some moderate respondents to discontinue the survey:
It's worth noting that those who completed the poll are also a self-selecting group. It's pretty commonplace in phone polls for respondents to simply hang up when asked loopy questions, and many of these questions qualify as loaded. As this poll progressed and questions like "do you think your state should secede from the union" came up, I'm willing to bet that the more rational Republicans just hung up, leaving the poll to be completed by those more divorced from reality.
Markos ("Kos") Moulitsas responded to such concerns in via twitter:
Cons[ervatives] arguing R2K poll might be skewed by "sane" Republicans hanging up b/c of crazy questions. R2K says no diff in response rates.
This may be true, but straight response rates might not tell the whole story here. A more important question is whether those respondents who started but failed to complete the survey were any different from those who did complete it.
Typically, the vast majority of those who do not respond are either not available when the pollster calls or refuse to participate immediately after answering the phone Those who hang up during the interview are a small component. For example, a report on the response rates for Marist's final 2008 New Hampshire primary poll includes only 70 incomplete interviews, compared to 2,990 dialed numbers that resulted in refusals or where the household could not be contacted. The cases that pollsters call "mid-interview terminates" could increase significantly without making a big dent in the overall response rate.
Moulitsas attempts to address that question with the following tweet:
In fact, R2K says that more than usual, "most of respondents enjoyed answering the questions once they agreed to participate."
Having been employed as a survey interviewer before, I know interviewers can get a feel for the reactions of respondents that might provide at least anecdotal evidence of whether it was difficult to keep some survey respondents on the line or whether respondents were offended by any of the questions. However, Daily Kos and Research 2000 could provide harder evidence that their results were not skewed toward the most extreme Republicans by providing the following information from their data on respondents who began but did not complete the survey:
By answering these questions, Daily Kos and Research 2000 would provide important context and help to address valid questions about whether their final sample was skewed in any way.
Thanks to Mark for contributing to this post
Advertisement
Advertisement
