The shifts in outcomes between the 2008 presidential and 2009 gubernatorial elections in New Jersey and Virginia were driven far more by shifts in voting preferences among groups than by changes in turnout across those groups. Only age groups show consistently substantial changes in relative share of the electorate. Vote preference, in comparison, shows quite large shifts between election years. While one narrative of the 2009 election was changing turnout motivation, this turns out to be substantially false. Instead, changes in candidate preference drove the Republican wins in both New Jersey and Virginia.

The chart above shows the direction and size of change in vote preference for nine categories and 27 groups measured by exit polls in both years. The arrows start at the 2008 vote and point to the 2009 vote. The length of the arrow shows the amount of change and the arrow shows the direction of change. The colors code the shift in majority vote from 2008 to 2009. Blue indicates a Democratic majority for the group in both years. Red represents a Republican majority both years. Purple shows the groups that switched from a Democratic majority in 2008 to a Republican majority in 2009. None of the 27 groups switched from Republican to Democratic majorities.

In Virginia, large shifts in preference came among 18-29 year olds, those without a college degree, independents, rural voters and males. Smaller but still interesting changes came among lower and middle income voters, both of which shifted from majority Dem to majority Rep.

The most talked about shifts are among partisans and ideological groups. The large 16 point shift from 49 to 33 percent Dem among independents has justifiably received a lot of attention. But perhaps as interesting is the similarity of partisan loyalty among Dems and Reps. Neither shifted by enough to make the length of the arrow stand out. Virginia Democrats actually increased their Dem support by a point, while Republicans came home by a small 4 percentage points more than in 2008. Clearly the independents drove the dynamics of the outcomes.

Among conservatives, there was a modest shift of 9 points more Republican support in 2009, and a 5 point shift among moderates. Liberals moved a single point more Democratic.

By contrast, the shifts in share of the electorate were quite modest, as seen below.

By far the largest shifts are among the various age groups. The 18-29 year olds dropped 11 points, from 21 to 10 percent of the electorate. Those 30-44 also declined a bit, from 30 to 24 percent. These were matched by gains of 9 points among 45-64 year olds and of 7 points among those 65 and older. Age is one of the most potent predictors of turnout, and as this chart shows, one of the most dynamic from 2008-09.

The other two groups with interesting shifts are the rise in share of the electorate among conservatives (up from 33 to 40 percent of voters) and the similar decline in turnout among Democrats, from 39 to 33 percent.

Not only are these shifts substantial, but they also stand out against the very modest shifts in share of the electorate for all other groups. Many of the arrows have invisible lengths, indicating very small changes of one or two percent.

In New Jersey, we also see large preference shifts and even smaller turnout shifts.

The giant preference change in New Jersey is among independents, the same as Virginia. NJ independents took a massive 21 point shift from 51 percent for Obama to just 30 percent for Corzine. Also as in Virginia, Republicans came home to their party a bit, from a 14 percent defection rate for Obama to just 8 percent defection to Corzine. Democrats meanwhile barely budge, down from 89 to 86 percent Dem.

There were other substantial movements in vote preference, among 30-44 year olds, moderates, whites, hispanics and males. In short, many groups in New Jersey made substantial movements away from Democratic votes.

By contrast, the makeup of the New Jersey electorate changed a bit among age groups but hardly at all for virtually all other groups.

As with Virginia, there were declines in share of the electorate among 18-29 and 30-44 year olds and compensating increases among those 45 and above.

No other group comes close to such large changes in size. Several change by exactly zero (the open circles in the chart) and most others have lengths too small to see in the plot. The nearest exceptions are a decline of moderates of five percent and a corresponding five point rise among conservatives.

The bottom line for both states is that turnout changes were mostly about the age structure of the electorate. Younger voters are more responsive to short term stimulation, and in 2008 that translated to relatively large turnout, while in the absence of that stimulus in 2009 the more stable commitment to voting among those over 45 advantaged that group.

The shifts in preference in both states were significantly larger for many more groups. Preferences are driven by candidates and issues and those were the primary drivers of the change in outcomes from 2008 to 2009.

Below are alternative looks at the data, comparing the share and the vote for each state. These give a better look at the entire set of groups, but it is harder to compare magnitude of changes along the diagonal line in these charts than in the arrow plots above.

600

I'm headed to bed but here are a couple of snapshots of election night.

First, above, New Jersey then and now. Whatever else you say about the race, Corzine lost support across all regions of the state and by relatively constant amounts. This "uniform swing" shows that he didn't just lose in Rep areas, or Dem areas, or urban centers. The decline in Corzine support was very widespread and quite even. An across the board loss.

In NY23, Hoffman generally outperformed McCain's vote in 2008, but not by enough to take the race. These are based on 87% of precincts reporting, so not quite final data.

This is a quick note on the new Siena poll in light of Scozzafava dropping out.

Can Owens pick up from Scozzafava supporters? Not so likely given these poll results.

Owens Fav/Unfav among Scozzafava suppporters: 19/50
Hoffman Fav/Unfav among Scozzafava supporters: 15/57

Looks like a wash with many likely to skip the choice of two disliked alternatives.

And the worse news for Owens is among independents:

Ind. Fav/Unfav Owens: 39/47
Ind. Fav/Unfav Hoffman: 47/33

More after daughter's soccer game!

BREAKING: Scozzafava suspends campaign. Divided party now unites. Demonstrates powerful appeal of right in Rep pty. What remains to be seen is if this appeal can win the general election. Amy Walters had a good catch at Hotline: Hoffman leading among independents. Thats good news for him. But other unknown is how Scozzafava's 20% split now. Are they loyal Reps or were they looking for more progressive issues and hence not likely to back Hoffman? With all attention on Rep split, has Owens given the district reason to back the Dem over a conservative? Fun!

NY-23 is the poster child for the conservative insurgency in Republican politics. What looked a few weeks ago to be a splintered party in NY-23 is looking more and more like a consensus developing in favor of the conservative candidate. If Scozzafava continues to collapse, Democrat Owens will no longer enjoy a fractured opposition but a united one.

Mark has posted extensively on the problem of polling in NY-23, so be sure to see his comprehensive post here. There are many problems with polling in this race, especially on the turnout side.

But setting those concerns aside for the moment, Hoffman's surge shows that an insurgency can succeed in winning over (so far) well over half of Republican voters. The string of Hoffman endorsements from Palin, Pawlenty, Pataki and others shows how seriously Republican leaders are taking conservative insurgents.

Given substantial investments in the NY-23 race by both party and independent groups, this might be seen as the best case for insurgents against the party establishment. The converse is that if Owens pulls it out, NY-23 will be a symbol of party fratricide. And if Scozzafava's support collapses, it will be a message of the power of party unity around conservative nominees.

I commented on this topic in a recent Christian Science Monitor story here, and spent a pleasant hour this morning discussing this and other insurgent challenges on Wisconsin Public Radio's Joy Cardine show. The audio is available here. At the time of CSM interview, Hoffman had not yet surged, something we talk about in the audio link.

The end of the summer saw some rebounding of President Obama's domestic approval ratings, and his overall job approval.  But it is now the foreign front that poses the greatest potential to damage the president's approval ratings. Afghanistan looms large.

Domestically, things have turned in Obama's favor. After falling through much of the spring and summer, support for Obama's handling of health care reform has taken a noticeable upturn since mid-August. Approval of his handling of the economy is flat or slightly up, and his overall job rating has risen 3 points or so since it's low point at the end of August. Even his worst approval rating, on handling the deficit, has stopped declining, though it remains below 40%.

But the problem is now with foreign policy matters and most specifically with Afghanistan.  Through the spring, even as he increased troop levels in Afghanistan, Obama enjoyed an approval rating on that war of near 60%.  But those ratings have taken a sharp dive over the last four months, even as domestic issue approval rebounded. Since June, approval of his handling of Afghanistan has fallen some 17 points, from about 60% to about 43%.

During much of the summer conversation focused on the damage a falling health care approval was doing to Obama's overall job ratings. Now the attention should turn to the potential damage of  whatever decision he makes about how to wage the Afghan war. Increasing troop levels, and therefore casualties, will not be popular among progressives in his own party, and will surely not win back any Republican support. But refusing to increase commitments there is likely to result in little visible improvement while maintaining American targets, and therefore casualties. While the president may have succeeded in bringing back some support for his domestic agenda, that was an easier sell within his party than will be the case with Afghanistan. The driving mechanism with Afghanistan is going to be casualties and perceptions of progress or lack thereof plus a base of ideological opposition to the war among Democrats.

The declining approval on Afghanistan has also begun to be reflected in declines in approval of his handling of foreign policy in general. (Handling of Iraq and Iran have relatively few recent polls, so are omitted from the figure.) While foreign policy approval has been a strength, generally running higher than overall job approval, that may not be the case much longer.

If the military decisions are daunting either way, the political implications are at least as difficult. A Democrat committing more troops to a war now eight years old, with troops stressed from repeated deployments, is going to be a hard, perhaps impossible, sell to his partisans. But with Republican support now down in single digits, the president can look for support only from Democrats and some independents. Those groups are quite reticent to support an escalation.

I talk about Obama, the economy, health care reform and the critical support for Blue Dog Democrats who may hold the fate of Obama's legislative agenda in their hands.  Alas, no charts this time.

The trend in opinion on health care reform has been a bit tricky of late. After a long and substantial rise of opposition, and an equally long but less sizable decline in support, we came to August, the month for the strange in politics. This year was stranger than most with loud and angry town halls, fearful politicians reluctant to meet constituents, cable news of gun toting demonstrators, an extended presidential vacation and the death of an icon. And what did all the rancor produce? Apparently, the sound and fury signified, surprisingly, a flattening of the trends in opinion. Opposition slowed its rise (not accelerated), and support halted its fall and by the second week of August, began a modest rise. And that before the president spoke to the joint session of Congress.

The picture is complex in the details, but essentially unanimous in the major points. Let me lay the details out a bit.

The chart above shows all the trend lines we might estimate for our health care questions, using 65 different levels of sensitivity for the smoothing. At one end, we smooth little and get a very (overly) sensitive fit. At the opposite end we smooth a lot and get a quite insensitive trend. In between is our standard estimator. The overlapping lines above make clear that the big picture is a common pattern over all 65 different degrees of sensitivity: A rise, then a flattening of opposition, and a smaller fall then a rise in support. Off to the right, you can see "bar codes" showing all the different possible current trend estimates from each of the 65 different levels of smoothing. The range of possible current trends is small, but there is a little overlap indicating some estimates may show support slightly ahead of opposition, though most show the opposite.

If you want to pick a fight, then you can pick a level of smoothing that is a little different from the others in some details. The chart below is the most sensitive estimator, one that is quite likely to chase noise, but which also picks up short term changes.

Now you can see a post-speech spike in support, and a smaller drop in opposition. Both are sharp, but both also show the beginnings of a reversion in the latest polls, with support beginning to dip and opposition steady or slightly rising. If you believe this estimator, then the speech mattered, but wasn't a game changer because the short term gain for Obama has now begun to reverse.

Or, you could use a still sensitive but not quite so erratic estimator:

And, Aha! the speech changed everything and the trend is now declining opposition and rising support! Well, maybe. But this rests on being sensitive enough to pick up the speech effect, but not so sensitive as to turn down based on those very latest polls.

Or you could use our standard estimator, which is "sensitive enough" but which tries not to chase outliers. It needs several polls to convince it that the polling trend has really changed.

Here we see an upturn, probably driven by the speech, but the blue line hasn't caught the red line yet.

If you compare the red line in the chart above, with the red line in our interactive chart today, you'll see a bug. For reasons not yet clear, the interactive chart is producing a straight line for the opposition trend despite the fact that our standard estimator is really the red line above. Mark mentioned this mysterious bug in a previous post here. We continue to hunt it and hope for a quick fix. (For the really nerdy, we have to stabilize the line when few polls are available, and sometimes it seems the "stabilizers" kick in when they shouldn't. Probably due to the very first poll which is far from the rest. Take it out of the interactive chart with the filter option, removing HealthDay/Harris, and you get estimates much closer to the standard trend above.)

Notice that the blue line above and the corresponding black line in our interactive chart are very similar. So the interactive chart is doing the right thing with one line but not with the other. The interactive chart overshoots putting opposition at 49.9% while my estimate above is 48.5%. Meanwhile support is 45.4% on the interactive chart and 45.8% here.

Finally, what if we intentionally over-smooth-- ignoring the day-to-day noise for the long term "fundamentals" trend:

The same basic story is apparent. Opposition has grown but is now slowed to a near halt. Support reversed its decline sometime in August and has begun an upturn.

And my big point is that this is essentially the picture you see in all these different trend estimates. The details are slightly different. A bump here and a drop there, and the precise estimates of support and opposition differ by as much as 2 points up or down. But the big picture is that opposition ramped up significantly through June or July but has recently slowed or stopped. Support fell less precipitously but has been working back up for a month (despite or perhaps because of the circus coverage in August.) We could pick a chart to fight over the details, but we shouldn't. It is the big picture of public opinion that is important here. Within a couple of points, opinion is evenly divided. The White House has gained a bit of momentum, but will be challenged to lower the opposition numbers, not just raise the support numbers.

Here is a chart of how the current trend estimates depend on the degree of smoothing. The range of opposition is 47-49 and the range of support is a little wider, 45-49.