Mr Obama, rest easy. Indian students have hit rock bottom

By Lant Pritchett Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning. — Winston Churchill, November 1942 The PISA 2009+ results are the end of the beginning. For the last decade there has been a debate. Some argued the levels of learning inside Indian elementary schools (primary and upper primary) are a national scandal and a threat to the future of India’s society, polity, and economy. Others appeared to believe that the main, if not only, problem with Indian schools was that not enough children attend them and that with more money and more of the same, all would be well. [caption id=“attachment_175129” align=“alignleft” width=“380” caption=“In mathematics, of the 74 regions participating the two Indian states finished again, second and third to last, again beating only Kyrgyzstan.AFP”]  [/caption] The last five years saw a relentless accumulation of evidence about the crisis of learning. The establishment has tried to deny, deflect, and dismiss the evidence on learning. Eventually the government of India agreed to participate in the PISA (Programme for International Student Assessment) - but only for two states, Tamil Nadu and Himachal Pradesh - and both sides agreed PISA was the litmus test. The PISA 2009+ results, which are both official and are beyond gain-saying, are unspeakably bad. They confirm the worst of what anyone has been saying about the levels of learning in India elementary education. In reading, of the 74 regions participating in PISA 2009 or 2009+, these two states beat out only Kyrgyzstan. • In mathematics, of the 74 regions participating the two states finished again, second and third to last, again beating only Kyrgyzstan. • In science, the results were even worse, Himachal Pradesh came in dead last, behind Kyrgyzstan, while Tamil Nadu inched ahead to finish 72nd of 74. But just coming in last (if we can dismiss as a relevant comparator for India a tiny Central Asian state) does not convey the enormity of how bad these results were, as not only was India last, it was far, far, behind its aspirations, both at the bottom and at the top levels of performance. PISA expresses the levels of performance in two ways, an overall index number and the fraction of students achieving various “levels” of achievement. The PISA index numbers for each subject are scaled so that the typical OECD student is at 500 and the standard deviation across OECD students is 100. The testing of thousands of students allows the results to present not only the average but also the worst (5th percentile) and best (95th percentile) students do in each country/region. PISA also classifies student performance into “levels” that represent different degrees of mastery of the material. Table 1 compares India’s performance to three groups of countries. The economic superstars have successfully completed the transition from poor to rich economies in just two generations - Singapore, Hong Kong, Korea (China’s only results are just for the city of Shanghai, which are the highest scores of any region tested, but this is too a-typical to really be comparable) and India aspires to their sustained success economically. The current superpowers are represented by the USA and the OECD average reflects India’s aspirations as a superpower. The rising powers are represented by the BRIC countries of Russia and Brazil which reflect the rise of the emerging markets. Compared to the economic superstars India is almost unfathomably far behind. The TN/HP average 15-year-old is over 200 points behind. If a typical grade gain is 40 points a year Indian eighth graders are at the level of Korea third graders in their mathematics mastery. In fact, the average TN/HP child is 40 to 50 points behind the worst students in the economic superstars. Equally worrisome is that the best performers in TN/HP - the top 5 percent who India will need in science and technology to complete globally - were almost 100 points behind the average child in Singapore and 83 points behind the average Korean - and a staggering 250 points behind the best in the best. As the current superpowers are behind the East Asian economic superstars in learning performance, the distance to India is not quite as far, but still the average TN/HP child is right at the level of the worst OECD or American students (only 1.5 or 7.5 points ahead). Indians often deride America’s schools but the average child placed in an American school would be among the weakest students. Indians might have believed, with President Obama, that American schools were under threat from India, but the best TN/HP students are 24 points behind the average American 15 year old. Even among other “developing” nations that make up the BRICs India lags - from Russia by almost as much as the USA and only for Brazil, which like the rest of Latin America is infamous for lagging education performance does India even come close - and then not even that close. To put these results in perspective, in the USA there has been huge and continuous concern that has caused seismic shifts in the discourse about education driven, in part, by the fact that the USA is lagging the economic superstars like Korea. But the average US 15-year-old is 59 points behind Koreans. TN/HP students are 41.5 points behind Brazil, and twice as far behind Russia (123.5 points) as the US is from Korea, and almost four times further behind Singapore (217.5 vs 59) that the US is behind Korea. Yet so far this disastrous performance has yet to occasion a ripple in the education establishment. [caption id=“attachment_175126” align=“aligncenter” width=“460” caption=“Table 1”]  [/caption] I have emphasised mathematics because many believed math was an Indian strong suit. The results for reading and science are similarly bad. Table 2 shows science results in a different format, which shows the proportion of children in various categories of performance. There are three points:

1. “Below level 1” doesn’t even have a description as it implies that so little proficiency is demonstrated it is impossible to distinguish from not knowing anything at all. In the USA, even with its socio-economic and racial inequalities and language inequalities and its failing inner city schools, only 4.2 percent are in this category. In HP 57.9 percent of 15-year-olds in school cannot be distinguished from not having learned any science at all and in TN 43.6 percent all in this category - 10 times as many as the USA.
2. PISA considers “level 2” as the minimum level that provides the science competencies that will enable them to participate actively in life situations related to science and technology. Since more than 80 percent of students in both HP and TN are level 1 or below this, most students in these states have reached age 15 ill-equipped for the century they will face.
3. While a thin elite that competes for the few highly selective technical institutes are globally competitive, this is a tiny fraction of the population. The estimate of the fraction of TN or HP students at level 6 in science proficiency was zero. Their estimate of the fraction at level 5: also zero. Of course this does not mean there are not such students in these states, of course there are, just that from the samples available in the study the best estimate was so small as to be indistinguishable from zero.

 These results on PISA 2009+, while tragic for what they imply for Indian youth and perhaps shocking to newcomers to this subject, come as no surprise to those who have been working on basic education in India: • Das and Zajonc (2008) used results from Orissa and Rajasthan to create indices on mathematics performance similar to those of TIMSS (Trends in Mathematics and Science Study) and found these states near the bottom of the global rankings. • Educational Initiatives carried out an 18 state study using sophisticated testing instruments and found levels of performance on TIMSS comparable items that were stunningly lower. For instance on the open ended question “Write a fraction larger than 2/7” less than 30 percent of Indian students in standard 8 could answer correctly compared to more than 70 percent internationally. • The APRest study led by Karthik Muralidharan and Venkatesh Sundararaman in rural AP asked the same questions of students in grades 2 to 5 and found very slow rates of learning progress. • The results year after year from the ASER ( 2010 2009) study supported by Pratham find that significant fractions of students in Standard 8 cannot master even Standard 2 curricular basics. In rural areas nationwide a third of children in grade 8 could not do a simple division problem and almost 20 percent could not read a level 2 text. The 2011 results, due out in a few weeks will show continued stagnation or even retrogress in learning. • Numerous studies by MIT’s JPAL, World Bank, NCAER/University of Maryland and other researchers found levels of performance that were shockingly low compared to curricular expectations. These PISA 2009+ results are the end of the beginning. The debate is over. No one can still deny there is a deep crisis in the ability of the existing education system to produce child learning. India’s education system is undermining India’s legitimate aspirations to be at the global forefront as a prosperous economy, as a global great power, as an emulated polity, and as a fair and just society. As the beginning ends, the question now is: what is to be done? Ajay Shah’s Blog