(Original release date: 2009) Think about the last study you conducted using an online panel. Was your questionnaire the first one the respondents had completed that week, or the tenth? Was your questionnaire the first one the respondents had completed that day, or the tenth? Did the panel company slip any additional questions into your study without your approval? Are you positive your respondents actually came from the panel with which you contracted?
Online panels are not created equally. And we can prove it. We had mystery shoppers sign up for a variety of online panels for a month, and respond to questionnaires as regular consumers (answering all questions truthfully). This report will tell you about:
- the panel that gave the average panel member almost 60 survey invitations per month
- the panel that would let your survey be completed by someone who has just spent multiple consecutive hours answering other questionnaires
- the panel that inserts their own questions into your studies without your approval
- the panel that invited respondents to participate, and then regularly closed the study just a few hours later
- the panel that regularly shuffled their respondents off to other panels to complete surveys
Fortunately, we can also tell you about some panels that treated their panel members with respect, did not overuse them, and did not use some of these highly questionable practices.
There are good panels and truly horrendous panels out there – and which one you select for your project can make a huge difference in the quality of the data you get. If you’re a researcher serious about getting high-quality results, it’s uncertain which is more frightening: Dirty Little Secrets of Online Panels or The Blair Witch Project. Don’t read this report by yourself in a dark room.
Please e-mail ron@greymatterresearch.com for a free copy of Dirty Little Secrets of Online Panels. (Also see our 2012 report More Dirty Little Secrets of Online Panel Research.)