Exit polls 2017: Why solid data collection rather than methodology is key for higher accuracy

Exit polls for the most awaited 2017 election are out. Different agencies have come up with different results and will keep the voter guessing and glued to TV sets on 11 March. Exit polls are normally a better barometer than opinion polls and if not quite accurate in terms of seats, they do suggest the direction of the trend. However, of late they have added to the confusion.[caption id=“attachment_2751960” align=“alignleft” width=“380”]  Counting time. Reuters.[/caption] One agency C-Voter has predicted hung assemblies in four out of five states. Only in Punjab, it has called out AAP as the winner. There too, it has predicted same vote share for both AAP and the Congress. Axis My India which got Bihar right got Tamil Nadu wrong. Today’s Chanakya which got Lok Sabha right got Bihar horribly wrong. Both are rooting for BJP in Uttar Pradesh. Confusing results: Look at the ranges In Uttar Pradesh, BJP is the single largest party across the polls except for National Dastak which is predicting BSP victory. Axis and Chanakya predict a sweep for the party. C-Voter and CSDS predict hung assembly. Chanakya, in fact, predicts vote shares for parties similar to 2014 levels.

In Uttarakhand, majority polls predict BJP win. This is in line with the trend the voter has followed since the formation of the state of alternate governments every five years. Only C-Voter predicts a hung assembly with ‘others’ to play a key role.

In Goa, most polls predict BJP to be the single largest party with odd two-three seats short of the majority. Hence it has the momentum pulling together independents to form the government. AAP is not doing that great except for one poll which sees it getting seven seats. There is an outside chance of AAP+INC+Others forming the government in Goa.

In Punjab, the trend is very clear. Akalis and BJP are losing the state and in fact getting a beating. However, who will win Punjab, is not clear. One poll each predict AAP and Congress win. Two polls predict a tie with 54-55 seats for each party. AAP has historically been underestimated given Delhi 2013 and 2015 results. If not for AAP, Captain Amarinder Singh would have won easily riding on anti-incumbency.

In Manipur, BJP is expected to put up a great show in line with its outreach in the North East. However, polls are divided on who’s winning. This along with Punjab is a state which could go either way.

Comedy of errors in poll summary As in the past, exit polls offer something to cheer about to each party. Though more cheers this time are for BJP as shown below.  Why the disparity in results There are multiple reasons for the disparity. While pollsters go about scientific sampling, voters may or may not respond in the same scientific manner. For example, some voting groups may be less fearful of participating in such surveys, others may not. Further, some if not many voters may actually lie (for whatever reason) to the interviewers. The pollster then needs to correct for this and very often uses historic information. Without getting into the science of it, using historic information is fraught with many problems – turnout, new voters, and consistency of polling booths. In one of our surveys, we were struggling to distinguish if the high percentage of upper caste voters were due to sampling issues or improved turnout. There are ways to correct them but that is where all the problems begin. There is a lot of reliance on historic data that may or may not be accurate or even relevant sometimes. In India, given the population growth and inconsistency in polling station coverage, previous elections data is far less reliable than say in the UK where the pollsters have been analysing large samples (200 per polling station) for the last 16 years with small changes in population and first time voters. The accuracy levels of exit polls in the UK are quite high. Given the above complexity, each pollster ends up with different results even if they go to the same polling station. Pollsters and their background The list of pollsters has expanded significantly in line with the global trend. Social media has increased interest in elections and this, in turn, has fuelled the emergence of companies focussed on politics. In the past elections, about 5-6 agencies provided forecasts. In Uttar Pradesh election in 2017, some 13 agencies provided a forecast. This is primarily supported by massive expansion and fragmentation of news media and by the simple fact that vote counting in India does not happen on the same day as the voting. While one should not question the mindset of pollsters during exit polls, one cannot say the same about opinion poll surveys. Exit polls remain in the mind of viewers and clients for a longer time than opinion poll surveys. There is therefore much more pressure to deliver accurate results when it comes to exit polls. That is where the background of the pollsters matters a lot. On an average, we have observed (from our industry connects) that pollsters with solid experience in data collection appear to be doing a much better job than those with solid experience in methodology and modelling. In India, good quality data produces better results than bad data with a PhD. Another complexity is the ownership of some pollsters. Pollsters who are owned or controlled by one party can be consistently producing favourable results for their party whether it is pre-polls or exit polls. The quality of research is a big issue nowadays. Many channels ask newbie agencies to do the polls for free, after all, they are giving them free publicity. This hampers the depth of research in our opinion. they are giving them free publicity. This hampers the depth of research in our opinion. Exit polls quench our curiosity and anxiety between the last day of the election and the counting day. Even if they are wrong, it eases the curious mind and probably ensures proper sleep, in that sense, it is no different from sleeping pills during days of anxiety. it is no different from sleeping pills during days of anxiety.