I am utterly amazed that so many intelligent and well educated people misunderstand, misinterpret, and misuse research data. (I have analyzed such data for nearly all of my adult years.) Case in point is the handlers of the Democratic candidates in KY for U.S. Senate.
In November, the DrDan team made hay over a Survey USA poll showing its guy 11 points up over Jack Conway. Last week, I got three e-mails about a Public Policy Polling (PPP) survey showing Conway up by four over Mongiardo. Jack Conway’s campaign now has the momentum, so it claims.
I doubt it that any claims can be made by any candidate at this stage of the race, but who knows?
Let’s then examine the two polls.
One source of possible problems with any data involves the polling team’s experience. Sampling a new population can be problematic, with unforeseen issues contaminating the results. All researchers (including myself) have been there and done that during their careers.
Survey USA has had a remarkable record in previous election years; PPP is a relatively recent polling organization. Could that be the reason for the differences in the reported Kentucky Senate polls?
Let’s then examine their accuracy in recent elections. In the 2009 Virginia and New Jersey gubernatorial races, Survey USA came the closest of all the many polls in predicting the actual outcome and PPP was second best (see Pollster.Com). These are two excellent polling organizations.
Let’s then closely examine the Survey USA and PPP data for the Kentucky Democratic Senate race.
Both organizations have good internal numbers, with standard error of measurements around 4.2%. What a minute. Wasn’t there only a 4% difference between Conway and Mongiardo in the PPP data? The PPP difference was then within the margin of error.
Also, both polls indicated a relatively high number (30%) of undecided voters. And, I suspect that very few of the sampled Conway” or Mongiardo voters are strongly committed to their candidate. Hey, it’s basketball time in the Commonwealth.
The two polls sampled a slightly different population of likely Democratic voters in the 2010 Senatorial Primary. The PPP data contained more females and a higher percentage of likely African-American voters than the Survey USA’s data. These two demographic groups tended to support Conway over Mongiardo.
Differences also exist in the relative influence that different regions have upon the total primary vote. PPP has Western KY and the Louisville area accounting for 61% of the vote (33% for Western Kentucky and 28% for the Louisville area). SurveyUSA has these areas accounting for 49% of the vote (23% for Western Kentucky and 26% for the Louisville area)
Both organizations, I think, have oversampled potential voters from the Louisville area. Based on a review of previous off-year elections(including the hotly contested 1998 Democratic primary for mayor), I believe that these voters should reflect a little more than 20% of the sample.
More perplexing is the wide variation between poll data in regional support for Conway and Mongiardo. The PPP has, for example. Conway tied with Mongiardo in Western Kentucky; while Survey USA has Mongiardo leading in that area by nearly 30 points. Voter preferences do not change that quickly.
I wouldn’t make any bets on or investments in either candidate based upon the PPP and SurveyUSA data. The PPP results are inconclusive. There is too much variation between polls for my taste.
So why, oh why, should you take these polls seriously?