ProLog

I ran across an interesting observation this week about how the UK online polling service yougov.com conducts ‘back-to-back’ polls, or successive polls asking basically the same question only a few days apart.  The danger of this is that the experience of answering the first question will influence the answer on the second.  There’s a certain Heisenberg Principle at work here; once you’ve observed a phenomenon, you’ve changed its behavior in some way.

This is an unfortunately side effect of panel-based polling.  The most likely scenario is that the poller has crafted a question, only to realize it’s biased or otherwise flawed after the poll was conducted.  If he wishes to correct his mistake and repeat the poll, he’ll likely produce similarly skewed results since the respondents will simply repeat their previous response, without appreciating the subtle difference in the new question.

Of course, back-to-back polls are only a problem if you asked the same group of respondents both questions, as is the case with an opt-in, panel-based polling system like yougov.com.  In a traditional ‘random’ sampling where you can be relatively certain of different sample sets, you’re OK.  The downside of traditional systems however is that you loose the ability to correlate responses among your sample set.