ProLog

When only two thirds of Americans can name the Vice President of the United States or their own state’s Governor, it seems little use to do any polling of public opinion at all.  With public opinion formed largely by 10 second sound bites and Daily Show skits, it seems opinions can be molded to reach any conclusion a media outlet wants to support.

To combat the public’s ignorance of today’s complex issues, and the faulty polling data it creates, Stanford’s James Fishkin developed a polling method he coined Deliberative Polling.  In Deliberative Polling, respondents are first polled on a series of questions about the issues of the day, but then are asked to participate in a series of moderated discussions that explore the subject areas in a thorough but balanced fashion. The respondents are polled again and their answers are compared before and after the discussion.

Proponents of Deliberative Polling such as psychologist and noted eldercare authority Ira Rosofsky point out that a well-informed sample population can return drastically different responses than an uninformed one, particularly on such current topics as healthcare.  In fact the effect of the deliberative nature of this method can be measured by comparing the difference in the before and after polls.

There is of course, a catch.  While it may be possible to conduct a thorough investigation of a subject before re-polling the respondents, it is quite difficult to be simultaneously thorough and balanced.   In fact, a good rule of thumb in the polling world is that the less you discuss the topic with the respondent, the more balanced the questions are likely to be.  Every word you add to the polling question (or setup) increases the chance that you’ll introduce a bias into the poll.  Furthermore, there is the difficulty of unringing the bell.  Once the question has been asked, you have planted a seed in the respondent’s mind that is difficult to remove, regardless of how much discussion you conduct afterwards.

So while Deliberative Polling shows promise, we won’t be doing any of it at Proloquor.net.  Instead, we plan to let our polls take their natural course, and will resist the siren’s song of Informing the Public.  Let’s hope the public is up to the task.

David Frum, Bush(43) speech writer turned conservative blogger at frumforum.com (formerly NewMajority.com), posted an interesting interview with pollster Scott Rasmussen about his latest venture in automated polling.    Apparently bowing to the pressure to introduce polls commissioned by a third parties into Rasmussen Reports, Scott recently spun off Pulse Opinion Research, specializing in client-centric polling.

In the interview, Rasmussen announced that Pulse will introduce a service next month that allows anyone on the Net to commission a poll for as little as $600.  The polling mechanism will use Rasmussen’s standard automated telephone inquiry system.

While the system comes under routine fire by critics who question the fairness of automated queries, unedited questions from otherwise anonymous clients will likely only strengthen their arguments.  Still, its a bold step by one of the innovators of the polling industry.  If Pulse can provide more than the autodialer with consultation services on questioning and statistical analysis of the results, it would be the first step into true open source polling.

I ran across an interesting observation this week about how the UK online polling service yougov.com conducts ‘back-to-back’ polls, or successive polls asking basically the same question only a few days apart.  The danger of this is that the experience of answering the first question will influence the answer on the second.  There’s a certain Heisenberg Principle at work here; once you’ve observed a phenomenon, you’ve changed its behavior in some way.

This is an unfortunately side effect of panel-based polling.  The most likely scenario is that the poller has crafted a question, only to realize it’s biased or otherwise flawed after the poll was conducted.  If he wishes to correct his mistake and repeat the poll, he’ll likely produce similarly skewed results since the respondents will simply repeat their previous response, without appreciating the subtle difference in the new question.

Of course, back-to-back polls are only a problem if you asked the same group of respondents both questions, as is the case with an opt-in, panel-based polling system like yougov.com.  In a traditional ‘random’ sampling where you can be relatively certain of different sample sets, you’re OK.  The downside of traditional systems however is that you loose the ability to correlate responses among your sample set.

Frank Luntz, has been getting a lot of press lately, particularly on Fox, over his recently published book “What American’s Really Want…Really” (amazon.com).  Luntz is a conservative political pollster with a talent for identifying words and phrases that steer public opinion, usually over actual facts.  In the book, he conducts a fairly in-depth series of surveys, asking Americans how the relate to the following statements:

• “I’m mad as hell, and I’m not going to take it anymore.”: 72%
• “My kids will have a worse quality of life than I have.”: 57%
• “Live free or die.”: 88%
• “The 10 commandments are a good guide to live by.”: 89%
• “I want it all, and I want it now.”: 35%

If you’re one of the few that look at polling data as a tool (really), you must be scratching your head at these numbers.  While conservatives have taking these data to underscore the momentum of townhall gatherings, liberals could make exactly the same claims to bolster the current administrations plans.  If polling data is to be useful, it must look past rhetoric and phraseology and into the public’s will.