The two most recent times seem to be special situations that are likely to have little relevance for 2010. The 2002 mid-term election saw the president's party pick up a few seats, probably because the country was still rallying around the flag in reaction to the terrorist attacks the previous year. President Bush's approval rating was still high (Gallup showed it at 68 percent in a poll conducted Nov. 8-10, 2002).

And the 1998 mid-term election found voters quite dissatisfied with the Republicans' vote to impeach President Clinton, apparently a major reason why the president's party was able to pick up a few House seats. In both years, the Republicans enjoyed slim majorities, and the changes did not affect their majority control.

If neither of the two exceptions just noted appear to have much relevance to what might happen in 2010, the third exception (the first chronologically) is a different story. Taking place just two years after President Roosevelt came into office during the Great Depression, the 1934 mid-term election campaign focused largely on the New Deal measures adopted by Congress. The result was a net increase of 9 seats for the president's party - from 313 to 322. Democrats today no doubt hope that a similar debate about the economic stimulus bills and health care reform will be a positive inducement for voters next year.

That's where the differing poll results - depending on whether the pollsters use "likely voters" or not - provide an interesting story. The latest results (noted in my previous article) suggest the Democrats have an overall lead in the congressional house vote of from six to seven percentage points among the general public or registered voters, but are apparently in a dead heat with Republicans based on likely voters.

If indeed, the two-party aggregate vote is about even on Election Day, that would almost certainly result in major seat losses for the Democrats, though it's difficult to say how many. The aggregate two party vote (the total voting for Democrats vs. Republicans in the country as a whole) is not a perfect indicator of how well the parties fare in winning House seats. In 2002, for example, the Republicans won 54.1 percent of the two-party vote nationwide, but got 52.6 percent of the House seats. In 2004, they won a smaller percent of the two-party vote (51.4 percent) but picked up three seats, expanding their majority to 53.3 percent of the seats.

In 2006, the Democrats won 54.2 percent of the aggregate vote, and won about the same percentage of House seats - 53.6 percent. Two years later, their share of the national vote increased by only .4 percent (to 54.6 percent), but they gained 21 seats to hold 59.0 percent of the House seats.

Despite the inconsistency between the percentage of the national vote and the percentage of House seats won by the majority party, a tie vote nationally would likely cause the Democrats to lose a significant number of seats. The Democrats beat the Republicans in 2006 and 2008 by about 8 to 9 percentage points. Currently, not many polls suggest the Democrats will win by that margin in 2010.

Of course, with more than a year to go, much can happen to shape the political landscape. No doubt, the most salient domestic issues will be the stimulus bills and health care reform, and no one knows for sure what will happen to the latter. And then there is always the possibility of some major international event that could influence the elections.

Democrats may hope for a repeat of the 1934 mid-term election, but history tells us the circumstances will have to be quite unusual for that to happen. As indicated in Part I, given the expected low turnout, I'd be especially attentive to the preferences of "likely voters." If they indicate a Democratic lead of 8 to 10 points, that would be unusual indeed.

In a recent conversation with Jay Leve, founder and head of SurveyUSA, I was alerted to a split sample telephone survey experiment he conducted last October in the San Francisco Bay Area. It was on the subject of the government's plan to bailout or rescue Wall Street.

Though the experiment dealt with an issue that is so last year (!), I'm writing about it now because it demonstrates how questions about specific policy plans can produce misleading results about the public's views of the broader issue - a classic case of not seeing the forest for the trees.

SurveyUSA Experiment

Jay tested four different ways of phrasing the bailout question, and each one found mixed to slightly negative results. But then two follow-up questions starkly contradicted these results to suggest a clear majority of the public was supportive of the bailout efforts.

While the SurveyUSA experiment tested four different ways of wording the bailout issue, three are rigorously comparable, and so I mention them first. I'll come back to the implications of the fourth version later in this post.

Each of the following questions was asked of a split sample of just over 500 respondents:    

1. The government may invest billions to try and keep financial institutions and markets secure. Do you think this is the right thing for the government to do? The wrong thing for the government to do? Or, do you not know enough to say?
2. The government may spend billions to bail-out Wall Street. Do you think this is the right thing for the government to do? The wrong thing for the government to do? Or, do you not know enough to say?
3. The government may spend billions to rescue Wall Street. Do you think this is the right thing for the government to do? The wrong thing for the government to do? Or, do you not know enough to say?

Was the Public Ambivalent?

With these results, one would have to conclude that the public was at best ambivalent toward a bailout or rescue of Wall Street. Given the sample sizes, Form A results are significantly different from those in Form C, suggesting "rescue" is the most negative way to phrase the issue and "invest" is the most positive way.

But then come two additional questions that throw a completely different light on the issue. The question below taps into a general feeling about the issue, and finds that people seem to want Congress to do something - and that they're more afraid Congress will do too little than too much.

The problem with comparing the above question to the other three is that this one doesn't explicitly allow for "no opinion," while the first three questions do.

Public Supports Bailout

But the next question is comparable in offering an explicit "don't know" response, and it also suggests that a clear majority of the public wants Congress to do something.

Note that despite offering an explicit "or don't you know enough to say" option, this question shows a clear majority in favor, with only 30 percent opposed.

Had we depended on only the first three questions, which were asked of separate split samples, for our understanding of the public, we might well have concluded that whether it was "bailout" or "rescue" or "invest," the public was either evenly divided about a bailout or leaning against such an effort. But these last two questions suggest that less than a third of the public was opposed to an economic rescue/bailout plan in principle, while a clear, though small, majority (54 percent) of people were in favor.

New Complications

When we look at the results of the fourth version of the split sample experiment, we find even more support for some type of rescue plan. This version included three substantive options (compared with just two for the other three versions - which is why I'm treating this question differently from the first three versions), followed by the explicit option of not expressing an opinion.

This version shows the least support for the current plan, but its second option - to pass a "different" plan - suggests that more than seven in 10 respondents are in favor of some rescue plan (31 percent the current plan, plus 40 percent for a different plan). These results also suggest that only 13 percent (rather than 30 percent as shown in Table 3) are opposed to some kind of effort.

Important Lessons

These results, based on a telephone survey in the San Francisco Bay Area, demonstrate how variable the survey results can be - even when the survey is about an issue that is the subject of much media attention. The first three versions of the bailout question could reasonably be interpreted to suggest that the Bay Area public was either ambivalent or negative about any bailout plan, while the question reported in Table 3 gives the opposite impression - that the same public was in fact looking for some action by the federal government.

The important lesson: Sometimes, asking about specific plans can blind us to the larger issue of whether some type of action is still desired.

The last question reaffirms that when questions have two options in favor of a policy (pass the current plan, or pass a different one) and one against (take no action), the "opinion" that is measured can be quite different from when a question has just one option in favor of a policy and one opposed.

The last question can also be interpreted to have two options against the "current" plan - only one option says pass the current plan, while two options are against it (pass a different plan, and take no action).

This doesn't mean that three options should never be offered. The example does reaffirm, however, that the more options that are offered in general, the less similar the results will be to questions that offer only two options.

Note: My thanks to SurveyUSA's Jay Leve for pointing me to his very insightful experiment.

In his post on "Hired Gun" Polling, Mark Blumenthal suggests the need for pollsters to "distinguish between questions that measure pre-existing opinions and those that measure reactions."

He makes an important point. Much of what pollsters offer to the world as "public opinion" is in reality hypothetical, based on giving respondents information that many in the general public may not have and then immediately asking respondents for their reaction to that information.

Such results can be illuminating, but pollsters recognize that feeding respondents information means the sample no longer represents the American public and what Mark calls its "pre-existing" opinion. Unfortunately, many pollsters fail to acknowledge the hypothetical nature of such results, and instead treat them as though they represent the current state of the public's views.

The problem with this kind of approach is illustrated in the case that Mark discussed in his post, dealing with the "card check" bill, the proposed Employee Free Choice Act (EFCA) concerning the authorization of unions in the workplace.

The vast majority of Americans, one can reasonably assume, have little to no knowledge of the provisions of the bill. Thus, to measure "public opinion" on the issue, pollsters feel they need to tell respondents what the bill is all about. A Republican pollster explained the bill one way, a Democratic pollster another way, and - to no one's surprise - they ended up with a "public opinion" that reflected their respective party's position on the issue.

While one may argue the relative merits of the questions used by the two pollsters, the main point is that informing the public of any major policy proposal is intrinsically biased. Pollsters have to decide what is important among all the various elements of the proposal, and they can often come up with quite different conclusions. This problem applies to public policy pollsters as well, who - we can reasonably assume - have no partisan agenda, but who nevertheless can produce what appear to be partisan results.

Such problems have multiplied with the recent public policy polling on the bailout proposals for Wall Street and for the auto industry, and on the stimulus plan being considered by Congress. Most pollsters assume the public has little specific knowledge of such proposals, and thus pollsters provide respondents specific information to measure the public's (hypothetical) reaction to the proposal.

When CNN described the proposal to bailout the auto industry by characterizing it as "loans in order to prevent them from going into bankruptcy," in exchange for which the companies would produce plans "that show how they would become viable businesses in the long run," it found a 26-point margin in favor (63 percent to 37 percent). But when an ABC/Washington Post poll only a few days earlier had mentioned the word "bailout" in its question and did not refer to plans leading to the companies becoming viable, the poll showed a 13-point majority against the proposal (55 percent to 42 percent).

Again, one can debate the relative merits of the two questions, but the tendency of pollsters is to say that each set of results provides different insights into the dynamics of the public's views on this matter. In short, each provides a picture of potential public reaction to the proposal, if the proposal is framed to the general public in the way each polling organization presented the issue to its respondents.

That distinction is generally lost in the news reports. Each polling organization instead announces its results as though they reflect the current views of the public, over which the polling organization had no influence. But the reality is that the polling organization inevitably shapes its results by the very way it presents the issue to respondents.

As Mark argues, such "reaction" polling has a useful role to play in the public discourse on public opinion. However, it's also important that pollsters make clear that their results do not reflect "pre-existing" opinion (opinion before the polls were conducted - though one might instead use the word "extant" opinion), but rather hypothetical opinion under restricted conditions.

It used to be that newspapers made a formal distinction between "hard news" articles and "analysis" articles - clearly labeling the latter as such. That procedure doesn't seem to be followed these days, but it may be a useful analogous model for pollsters. Perhaps, in a similar way, pollsters can devise a method to formally separate their reports of potential "reaction" public opinion from existing public opinion.

I can envision, for example, one article in a newspaper that describes how many (or how few) people are actually aware of an issue and how many express ambivalence about the matter, while another article could explicitly describe how the public might react if the issue were universally framed in one way or another. Pollsters have made such distinctions sporadically, which is why we know that the public is more likely to support "rescuing" the auto industry than "bailing it out."

Still, the need for a formal and widely accepted method of distinguishing "reaction" questions from those that measure existing opinion needs to be found, if pollsters are to avoid the confusion that occurs when highly reputable polls produce wildly contradictory results.