Can a Shopping Voucher Skew Survey Results?

Unsuitablog:

A couple of weeks ago I – or rather our house – received a letter from an organisation called the National Centre for Social Research, inviting us to take part in an annual survey called British Social Attitudes. I like filling in surveys when I have time because it lets me think more about what I believe, and also provides a nice opportunity to screw up systems of categorisation, having found out some time ago that I neatly fitted into not a single one of the geodemographic categories that corporations and governments like to believe we should all be defined by.

This letter contained a little surprise, though; and not a very pleasant one. From the envelope fell a voucher for £5, going by the name of “Love2shop“. This voucher, like all gift vouchers, specifies precisely where the face value can be redeemed, and the range of outlets is ok…if you can see past the gruesome “Love2shop” monicker.

I hate shopping.

But personal opinions aside, what is far more worrying is that the British Social Attitudes survey is meant to be an objective and robust record of British attitudes (at least of those people who have mailing addresses). The letter states:

British Social Attitudes is the most respected and accurate guide to what people in Britain think about a range of issues, so you may have come across facts from our reports in the newspapers or on television. For 27 years our findings have been used by government, top universities, charities and the media to inform policy and insight.

By my reckoning, if such a survey is to fulfil all that then nothing can be allowed to skew the randomisation of the subjects, nor their mindset when answering the questions. The problem is that even though the addresses selected for survey, and the person selected for interview in that address, is statistically random, no one is obliged to take the survey. The questionee has to accept the invitation, so there is a significant element of self selection involved in the survey. Now, who do you think is more likely to be persuaded to take part in a survey upon receipt of a Love2shop voucher: someone who considers themselves to be a “Consumer” or someone who does not?

If this outcome was intentional then I would say that someone was trying to doctor the survey in favour of a consumer society.

I contacted NatCen to find out why, given the apparent objectivity of the survey, someone had thought it would be a good idea to include a shopping voucher as an incentive. The nice researcher I eventually spoke to seemed keen to listen, and indeed took what I said quite seriously; but also said categorically – as far as she knew – that there were no extenuating circumstances: the voucher just seemed like a nice idea at the time and, as far as she was concerned wouldn’t affect the outcomes of the survey. I begged to differ, pointing out that the methodology was already bound to account for the likelihood of different types of people to take surveys, and therefore if the effect of the shopping voucher had also not been accounted for then the survey would be void.

She promised to let someone know.

As an aside, I was also very keen to know the contents of the survey itself, ostensibly to find out whether the questions and the way in which they are asked skewed the outcome towards any particular outcome – one, for instance, that may be desired by a society obsessed with industrial and economic growth. Surprisingly, she not only sent me a link to the survey (you can download the PDF here) but, on first glance, it seemed to be a fairly well conducted survey – albeit something that might have been commissioned by the Daily Mail.

So what of the voucher? Well, sadly it seems to reflect the blanket attitude that everyone in public life seems to have smothered humanity with: we all want to shop.

Read the original story at Unsuitablog