This article is authored by Sridhar Rajagopalan, Co-Founder and Chief Learning Officer, EI (Educational Initiatives)
India has decided to participate in PISA in 2021. A small group of scientifically selected students from Chandigarh, the Kendriya Vidyalayas and the Navodaya Vidyalayas will write the test that year. In this article, we use data and insights from other assessments to analyse why Indian students do not, in general, perform well in international assessments like PISA. (Needless to say, many Indian students do perform well on these and other tests – most of our comments in this article refer to the vast majority who do not.)
Why do Indian students struggle on tests like PISA?
India participated in a modern international assessment for the first time in 2009-10 when some students from Tamil Nadu and Himachal Pradesh appeared in the PISA test. However, we performed very poorly – ranking 73rd out of 74 countries that participated that round, finishing ahead of only Kazakhstan.
The result was so shocking that many people assume that it must have been an aberration. Some feel that the students may not have been prepared for the test. Others believe that the students may have been tested in English which they were not proficient in. (Actually, all students were tested in their medium of instruction.) Still others feel that the performance must have been poor because only government schools were tested – our private schools would have done much better. (But a well-publicised study by EI in 2006 and repeated in 2012 established that even students of our top schools would perform well below international average in grade 4.)
In discussions, many educational officials express the view that students performed poorly because the tests and the questions were not suited for Indian realities, especially in rural areas. For example, PISA has had passages on topics like genetically modified crops and electric cars – and that contextual disadvantage is why Indian student performed poorly.
However, that is not a convincing argument. While PISA questions may have different contexts, information about that context (for example, what genetically modified crops are) is always provided in the passage. Not just Indian children, but world over many children may not know much about these topics beforehand. One of the important skills tests like PISA assess is how well students can understand key ideas related to new information.
PISA tests the skills and knowledge of students in reading (language), mathematics and science and how well they can apply what those skills and knowledge to real-life situations. It is important to appreciate that as good tests go, PISA is not unusual in this respect. Another international assessment, TIMSS or even a good Indian test like our own ASSET use questions that have examples from unfamiliar contexts or which look different from questions students are ‘used’ to. That is what tests if children have really learnt a concept and can apply it in real life.
The question explains the basic principle of the working of a seismograph (which measures the intensity of earthquakes) and asks a question based on the graphs recorded during an earthquake. Even some teachers who saw this question commented that seismographs were not ‘in the syllabus’ in their schools! Needless to say, seismographs are unlikely to be in the syllabus of any country in grade 9, and that is not the point. Students are expected – and should be able – to understand the information (in this case that the earthquake’s unusually large vibrations will be captured by the graphs and use that to answer the question.) In our sample, only about 30% of students from private English medium schools who take the ASSET test, were able to answer the question.
The argument that the context of the questions is the reason Indian students performed poorly implies that our students would perform well on questions that have a familiar context. Figure 2 contains a question with a simple and familiar context. Less than 20% of over 30,000 class 9 government school students in an Indian state answered this correctly. (Most students think that the largest decimal number is D. 250.49 which is bigger than B. 250.6 because 25049 > 2506.)
The last example indicates that our students perform poorly in tests like PISA at least in part because of poor understanding of basic concepts. This is a sobering thought. In our experience this is NOT because students or even teachers are inherently weak, but more because of the nature of questions they are asked in the public (Board) exams and trained with in their school exams. In addition to other factors discussed below, if our children have to perform well on PISA, they must be trained to answer questions that require them to understand the question, figure out which concepts apply and work out and give the answer. This will not be an easy or quick process, which is why the sooner we start it off as an educational system, the better it is.
Let us look at all the reasons why assessments like PISA (and even ASSET) turn out to be difficult for most Indian students.
1. The mentality that questions can be only from the textbook or minor variants of textbook questions
Not only students, but even teachers and parents fundamentally seem to be believe that questions in Board and school exams must be of the pattern that students are familiar with, ideally matching questions in the textbook itself. Just search for ‘out of syllabus questions’ and there is no shortage of newspaper articles on parent protests about questions that are deemed unfair for this reason (see ‘Couple of ‘out-of-syllabus’ questions in SSC Maths II’ or ‘Out-of-syllabus novel question knocks them over’). No doubt the seismograph question above would be seen as ‘out of syllabus’ but what would one say then of virtually every PISA question?
The power and responsibility to change this mindset lies primarily with our Examination Boards. But they face 2 challenges – one, a push back from schools and parents (some believe at the goading of coaching institutes) if they try something different – such pushbacks quickly escalate into bad publicity for the government – and two, the lack of ecosystem support, for example, people who can develop such questions. Yet, the responsibility to present a strong case for change (‘Shanghai-like performance in PISA, anyone?) and build that ecosystem lies primarily with the Boards though schools, parents and indeed, all of us, should support these changes.
2. READING, READING, READING!
Most application questions are based on a context which is first explained in the question. This requires students to read through at least a few paragraphs to understand what is being asked. Our average student has very poor reading abilities and usually prefers to guess what the question must be by glancing through it.
3. Process of answering questions – pattern matching versus problem solving
Leading from the above 2 reasons, the strategy that students use to solve questions in typical Indian exams is very different from what is needed for tests like PISA. Firstly, it is quite easy for students to recognise question types in the Indian tests since all exams follow a set of patterns that they are familiar with or have prepared for (including through their coaching and tuition classes). Questions on probability in Mathematics are of certain types and they know the possible questions that will be asked from a particular passage in the English reader. The process of solving the paper, then, is one of rapidly identifying the pattern, recalling the steps in the process and writing those steps. Almost every question follows this recall / reproduce pattern. Needless to say, in this scenario, reading questions carefully is unnecessary at best and a waste of time, at worst!
The PISA or ASSET questions, as we can see, are very different. There is no pattern to match with. Rather, the problem itself has to be read and understood. Some of the information may be in a figure or graph. Not all of the information presented may be relevant, as is common in real life. The academic concept that needs to be applied is not clearly given – multiple concepts, possibly including those learnt in lower classes may need to be used. In the Slope-Face Investigation, for example, the student has to read and understand the experiment. The concept of uncertainty measurement needs to be understood, yet is only a small part of the problem.
4. Being put off by the unfamiliar and not proceeding further
When Indian students encounter PISA-type questions, many of them freeze at the first sign of the unfamiliar and decide that they have not ‘learnt this question type’ and cannot solve it. Many students abandon the question (or decide they will come back to it later) at this point. Note that the question itself may NOT be a difficult one if they cross this stage. This is true in the Slope-Face Investigation as well – it is very clear that of the 2 parameters being compared, one varies significantly across the 2 slopes while the other does not. For students who reach this stage, the question is not a difficult one.
In fact, it is seen that students perform more poorly on unfamiliar questions than genuinely difficult ones!
5. Genuinely low understanding of processes or concepts and even comprehension skills
And finally, like we saw in the decimal comparison example, actual learning levels and understanding of concepts is low. The fact that mainstream exams do not test concepts in a different or unfamiliar way reduces student exposure as well as teacher incentive to focus on genuine understanding of concepts.
Like in a PISA question, identifying the problem is a key part of finding the solution. Each of the above represents an entrenched, yet solvable problem in the Indian education system. Though there are no quick fix solutions, there are key levers available to create change. Changing the pattern of Board Exam questions – and teacher training starting with teachers from grade 5 or so are two strong levers in our control. The facts that few seem to disagree that the system needs change and most would even agree with these broad approaches are also huge positives. In the journey to an India where every child is learning with understanding, PISA is just the first milestone on the road.
