Editor's note: This article was originally published on 11 July, 2016. It is being republished in light of the Delhi High Court order to the Central Board of Secondary Education to retain the system of marks moderation, leading to the withholding of CBSE results for Class XII students.
79 = 95.
If the authors of this piece claimed the above was true, you’d laugh it off. But if we further claimed, 80 = 95, 81 = 95…, and so on till 94 = 95, you’d call us mathematically illiterate. We can assure you, we’re not.
But CBSE probably is going by the marking system followed for Mathematics — it has brought everyone who scored between 79 and 94 in the subject to the same parity score of 95. In some cases, those who scored above 95 have been dragged down to this pleasant-sounding score. In other subjects too, such glaring inconsistencies are aplenty.
Our numerous interviews with insiders*, from ICSE/ISC and CBSE boards, as well as analyses carried out on data covering a number of years and hundreds of thousands of candidates, corroborate that board exams in India, including those of national boards ICSE/ISC and CBSE, have turned into a farce — marks are randomly inflated, marking criteria is opaque, and luck plays too big a role.
Our analyses establishes that the statistical patterns of scores allotted are simply not as one would expect in public examinations with so many candidates. Owing to competition between boards as well the government’s desire to project a better image, school boards have turned into arbitrary marks-giving machines. An ominous warning of this comes from the fact that a single state board has managed to occupy 80 percent of the seats allotted so far in India’s top-rated commerce college, where normalisation across boards was completely eschewed.
Some arbitrariness in scores might be attributable to careless evaluation. But the overwhelming reason behind the massive statistical anomalies is the practice of score moderation by different boards. Usually, moderation is a justifiable practice involving slight upscaling or downscaling of marks, once answer sheets have been graded, to account for inter-examiner variability and swings in the year-to-year difficulty of question papers, so that yearly pass rates and score distributions don’t fluctuate wildly.
Unfortunately, in the name of score moderation, marks and pass rates are being arbitrarily increased — a practice commonly known as 'grade inflation' — which has rendered the board examinations statistically and qualitatively invalid.
In fact, the term grade inflation itself is misleading, and would be suitable only when students obtaining 100 marks, a few years ago were obtaining, say, 110 now. But that’s not the case. Since the upper bound is 100 (or grade A for CBSE class 10 exams) and everyone’s marks are being artificially increased, far too many students are now ending up in the higher 90s, which means 100 marks now would be equivalent of substantially lower marks a few years back. Besides distorting comparison across years, what’s worse is that concentration of students in the higher marks bracket blurs the line between good and bad performers from the same year — ruining their effort to secure admission in top institutes. Thus, we should shun euphemisms and term this practice what it really is — 'grade distortion'.
During our conversations with insiders, each of them pointed out the complete lack of transparency in marking scheme and moderation, and they unanimously agreed that the project/practical marks — 20 in ISC and 30 in CBSE — were simply for the sake of boosting scores, and had no relation whatsoever to the merit of the candidates.
A teacher, who has been ICSE chief coordinator (second highest in the board’s marking hierarchy), said that the marking scheme was unacceptably biased in favour of ‘keywords’. In what he said was grossly unfair to students, the marking scheme and keywords themselves change substantially from year-to-year, with little transparency or basis. Giving an example from Biology, he added that till a few years back the word “solvent”, and not “water”, was accepted as correct in the definition of the process of osmosis. But an abrupt and unannounced change meant the latter started being accepted as correct.
According to him, another issue with the marking scheme of any subject, is that it is almost solely the prerogative of a single person — the chief examiner of that subject. The paucity of time available to examiners means standards are loose. In an example that was highlighted at an ICSE meeting, an answer sheet was never opened and marks were randomly allotted on the front sheet. Further, in a well-functioning system, the chief examiner should randomly select answer sheets already examined by his deputies to make sure the marking scheme is being followed. However, this teacher said that there was no randomness in selection, and it was usual for chief coordinators to ask their deputies to present sheets of their choice.
In the most shocking example from his career, a big bunch of Biology answer sheets belonging to a particular state arrived soaked in rain, with the ink on them completely wiped out. Confused, he sought guidance from the chief examiner of the subject, who promptly asked him to assign average marks to all.
Another senior ICSE/ISC teacher and examiner said that often times the scores that appeared on the final result of a candidate whose answer sheet she’d graded were far above than what the candidate could have achieved even, given full marks in the project/practical. She admitted to being completely baffled by the frequency of such an occurrence. She also added that it was usual for moderation to occur at all levels, including that of chief coordinator and the chief examiner of the subject.
Another blot on the marking scheme, she added, was that spellings often didn’t get penalised as long as the phonetic sound of the misspelled word was the same. In yet another example of dependence on keywords, she added that an answer describing the process of respiration would get full marks if the words “taking in” and “oxygen” were present anywhere in the answer, irrespective of what the rest of it contained. On the other hand, a perfectly descriptive answer would get penalised if it didn’t have those two keywords.
Being a private body, the CISCE (the board that conducts ICSE and ISC exams), unlike the CBSE, is not under purview of RTI, which means they never show candidates the photocopies of corrected answer scripts. This increases the likelihood of neglect and arbitrariness in marking.
The state of CBSE doesn’t seem much healthier, though. A senior teacher, and now principal, at a CBSE school pointed out that CBSE had always been under tremendous pressure to project a healthy state of the country’s educational attainment. This pressure had been amplified by competition from other boards, leading CBSE to hugely inflate marks.
An administrator at a top-ranked CBSE school mentioned that CBSE had barred re-evaluation of answer sheets of most elective subjects because there was a dearth of qualified examiners for other subjects, and they were being arbitrarily marked, often putting those opting for these electives at a disadvantage. In support, he cited far better scores obtained by candidates on the same subjects in internal exams, as compared to board exams.
As mentioned earlier, our analyses revealed statistical anomalies in data for board exam marks. Below are some conclusions.
Jump in share of 90-percenters and overall scores
The share of 90-percenters in CBSE has jumped to almost 8 percent in 2016 — this fraction was slightly less than one percent in 2004.
The graph below shows how the median CBSE (Class 12) percentage, for all subjects combined, has changed since 2004. There has been a jump of nearly 8 percentage points.
In the CBSE examination, since 2009, an unbelievably high number of people score exactly 95, in various subjects. Also, CBSE and ISC are desperate to somehow pass candidates.
The histogram below depicts Mathematics scores in CBSE 2015. Notice the spike at 95. This is because, as mentioned at the beginning, everyone between 79 and 94 ends up at 95. Other spikes show similar “pull-up”. The missing bars between 18 and 33 (passing marks for CBSE) clearly depict the generous policy of “grace” marks — awarded solely to make candidates pass. A 10-year analysis of this data, for the period 2004-2013 may be viewed here.
For ISC, we found that over 99 percent of the candidates have been passing English for the past few years. We suspect this is simply because failing in English implies failing in the entire exam.
Specific regions are favoured arbitrarily: In English, CBSE favoured Delhi students by a huge margin.
Continuing with its love for 95 marks, CBSE allotted this score to over 18 percent students in Delhi, in English. But the rest of the country was less fortunate, with less than 1 percent scoring 95. Even in reputed schools in metro cities other than Delhi, very few have scored in the 90s in English. As reported by TOI, there is upward moderation of 12 marks for Delhi, but zero marks for others. Such regional bias exists for other subjects too.
In both the ICSE and ISC examinations, since at least 1997, it has been observed that certain numbers are altogether missing from the result.
In at least 20 years, a period during which a few million students have appeared for ICSE (class 10) or ISC (class 12) examinations, not a single person in the country has scored 93, 91,89,87,85,85,83 in the country, in any subject. There are totally over 30 such marks which have never been attained. This indicates a serious glitch in the moderation process.
Below is a histogram with scores obtained by students in Mathematics in ISC-2015. Notice the missing gaps in the graph. The biggest one right below 40 (passing marks for ISC) may be attributed to an ultra-liberal policy of grace marks, similar to CBSE’s.
Rising popularity of subjects with inflated scores
Certain elective subjects such as Physical Education and Computer Science are exceptionally high-scoring in both CBSE and ISC examinations. Often, over a third of the candidates score higher than 90 in such subjects. It is unlikely that such high scores reflect the true capability of the candidate. Over the years, these subjects have become increasingly popular, for reasons that have nothing to do with academic inclination. The percentage of takers for Physical Education at the ISC examination jumped from 5 percent in 2012 to nearly 16 percent in 2016. This has come at the cost of subjects such as Economics, which is less scoring. Such preference is nothing but a warning against perverse incentives. Internationally, too, there is evidence of the same.
The insidious phenomenon of grade distortion isn’t unique to India. It has been widely reported amongst US universities. In 2014, Princeton had to abandon its policy of capping the share of A-graders under pressure from students who found it hard to compete with those from other universities getting runaway grades.
Perhaps it is no surprise that premier institutes in India are trying to wean off their dependence on board marks. The latest IIT-JEE pattern has done away with board marks as a criteria, since, as per Ashok Mishra Committee’s report, they pose a “big challenge” mainly due to “difficulties in normalisation”. Other institutes like BITS, Pilani have introduced their own entrance tests, leading to a big churn in the geographical representation of their students, indicating that certain boards made it easier for students to gain entry.
Grade distortion needs to be looked at as a serious public policy problem. It is the educational equivalent of what is called dole/easy money in economics, something that encourages lesser effort on the part of students, resulting in further deterioration in the nation’s human potential, and eventually its future. Moreover, it is ruinous to the idea of meritocracy — better the effort, sweeter the fruits. Not quite, with grade distortion around.
Prashant Bhattacharji created the education cum data portal The Learning Point and the data analysis above is based on data from here. Prabhat Singh works for the Andhra Pradesh government and freelances as a journalist. He tweets @singhK_P.
With inputs from Debhargya Das.
*none of the interviewees agreed to come on record.
Published Date: May 24, 2017 10:39 AM | Updated Date: May 24, 2017 10:41 AM