Pre-election polls: Do they work, do they influence voters, and should they be banned?

by Rajeeva Karandikar, Director, Chennai Mathematical Institute.

I have been a psephologist for about 16 years and have had a fair
amount of success in predicting seats in an upcoming election. Here is
a post-mortem of what I said and what happened in the current round of
elections.

From 2005, I have been working with CNN-IBN and Centre for Studies
in Developing Societies (CSDS). CNN-IBN engages CSDS for the
survey. CSDS does a great job of running surveys `by the book’. I use
vote share data from the survey to come out with seat projections,
which CNN-IBN carries on air.

How did we fare in the recent state elections?

Seat count
predictions on air (CNN-IBN)

The outcome

These results show the power and the limitations of opinion poll
based projections. If one simply counts the number of cases out of 13
that the actual results are within the interval projected, the score
is just 4. However, one should see these as having correctly predicted
clear and decisive victories for the BJP in Rajasthan and Madhya
Pradesh. For Chhatisgarh, we predicted correctly that the BJP will
win, with a much smaller gap than Madhya Pradesh and Rajasthan. So I
would count all the three as being good predictions, with Chhatisgarh
being very good.

As for Delhi, we underestimated AAP support and marginally
overestimated BJP but we had the ordering right: Congress in third
position with the possibility of touching a single digit, and BJP as
the single largest party.

Vote share estimates
on air (CNN-IBN)

The outcome

Here, the survey has worked well, and the errors are generally
within the statistically acceptable range. The conversion from votes
to seats is a non trivial transformation – since one needs to estimate
the distribution of votes across the state in addition to the overall
percentage of votes in the state. This requires building an
appropriate statistical model. I will explain my methodology for this
stage in a future article.

One can see that in MP and Rajasthan, there was underestimation of
BJP votes by a few percentage points. The error in vote to seat
conversion went in the same direction, and as a result our prediction
was much lower than the outcome for the seats obtained by BJP. In
Chhattisgarh on the other hand, the survey estimated the gap between
BJP and Congress as 4% while the actual gap turned out to be less than
one percent. In this case, the error in vote to seat conversion and
the error in the vote share estimate cancelled out, and we got a
result that was bang on. Of the four, I was the least confident about
Chhattisgarh (and I had said so on air) since the estimated gap in
vote shares was small.

In my experience, the predictive power of any opinion poll that is
done a while before voting is rather poor. For one, any such poll can
only measure the mood of state or nation at the time of poll and
cannot estimate the potential change that can happen close to the
voting day. Some psephologists claim to estimate this change by
conducting polls at regular intervals and then extrapolate to get an
estimate of this change. However, this assumes there are linear time
trends in vote share, which is unlikely.

The other problem is the selection process that determines who in
the general population (that is sampled in a survey) shows up to
vote. The propensity to vote is not uniform across socio-economic
stratums of the society. One can try to factor these in but that can
inflate the error.

Exit polls are designed to take care of both these issues. However,
choosing respondents in a randomised manner as they exit the booth is
rather difficult and our experience has shown that it does not produce
a representative sample (as measured by the gap between the
socio-economic profile of the sample and population). Hence, we prefer
to do a post-election poll, where in the days following actual voting,
we do a household survey with sound methodology. In the current
round, various exit polls had got numbers close to the actual results
for MP, Rajasthan and Chhattisgarh. But for Delhi there was wide
variation. And in the past, there have been occasions when exit polls
had given an incorrect picture while we got it right with our
post-poll.

Going beyond forecasting, these polls are valuable in understanding
what was happening on the ground. As an example, the CSDS poll in MP,
Rajasthan and Chhattisgarh showed that the gap between BJP and
Congress vote shares was higher in rural areas. This diverges from the
common view that the BJP is an urban party. The CSDS website
gives the breakup of voting intention by various socio-economic groups, and
this is valuable knowledge.

Regulation of opinion polls

Do opinion polls influence voter behaviour? In each of the surveys
done by CSDS, one standard question is: `Who did the respondent vote
for in the last election’. This refers to the last Vidhan Sabha
election, if this is a poll for the Vidhan Sabha and the last Lok
Sabha election, if the current one is for the Lok Sabha. Almost
invariably the recall for whoever won the last time is much higher
than the actual votes, even when the winner from previous poll is
set to lose the current election.

Thus in 2011, a much larger percentage of voters seem to recall
having voted for the left front in 2006 though in 2011 they were
voting for Trinamool Congress. I may add that our estimate of the
Trinamool vote share and seats in 2011 was very accurate. The same was
true in Tamilnadu where, while voting out the DMK, a much larger
number of respondents seem to recall having voted for them.

We have observed this time and again across various states and
consistently over the last 15 years. The only explanation that I could
come up with is that there is a general tendency to go with the
winner.

This raises the concern that a political party may run a media
campaign claiming that it is ahead in the polls. This justifies
regulation (though not a ban). I feel this regulation could be self
regulation by the media, e.g. through the Press Council. The
regulation should require that each published poll reveals, in public
domain, the detailed methodology of sample selection, the sample size,
the socio-economic profile of the sample, the dates when sampling was
done, the names of the core team members who supervised the survey and
the methodology used to convert vote estimates to seat conversion.
All agencies that release such information should be open to an audit
by an expert group formed by an autonomous body such as Press
Council.

CSDS and I have been very open about our methods. The sampling
methodology is on the CSDS website, the sample size is always given on
air and the socio economic profile of the sample is also given on the CSDS website. I have written
about the vote to seat conversion in an academic article:
Predicting the 1998 Indian parliamentary election, Karandikar,
R. L.; Payne, C.; Yadav, Y., Electoral Studies; 21, 1; 69-89,
and have been talking about it in seminars.

Fortunately, over the years, the visibility of any one opinion poll
has declined, with so many agencies doing polls and making
contradictory predictions. In the recent Delhi elections, the
range of seats projected for the Aam Admi Party was from 6 to 31 out
of 70 seats! Hence, the salience of this debate has probably declined
greatly.

I do believe that there is a feedback loop: if all surveys point in
a certain direction, at least some voters tend to get influenced. But
this is no reason for a ban. After all, newspapers and TV channels
also talk about their assessment of the political situation and if
they all seem to point in a certain direction, this too has an effect
on the electorate. In addition, under the Indian legal system, while the
government can easily do censorship on television, this is harder
with newspapers and more generally on the Internet. Hence, even if a
ban were desirable, it is not feasible.