Fifteen thousand students were affected back in June by mistakes contained in a question on paper two in Higher Level Maths. With results due out tomorrow morning, and a report for Education Minister Ruairí Quinn nearing completion, teacher Eamonn Toland – managing director of TheMathsTutor.ie – reminds us what happened, and outlines how the State Examinations Commission can make sure it never happens again.
THE MAIN ISSUE was that the trigonometry question was set out with too many items of information, thus creating a scenario that was not mathematically possible. In effect, examiners attempted to describe a triangle using a set of measurements that can’t actually co-exist in a single triangle. This meant that the question was fundamentally flawed. Students answering the first question posed about this scenario could arrive at different answers using legitimate mathematical methods. Follow-on questions would also be impacted. (For a video overview of the problem, click here).
So, how does this affect students?
Well, the stronger students would have found it unusual to be provided with four measurements to describe a triangle. Of course a triangle has three angles and three side lengths, but usually you’d expect to be given two side lengths and an angle, or two angles and a side length, and that would be enough to figure out the other measurements. Being given four measurements should set off alarm bells, especially for a strong higher level student.
Because of this, students might have chosen to solve the first part of the question using very basic trigonometry, and have then decided to quickly check their answer by using the same method but going at it (literally) from another angle. If they did this, they would be very puzzled, because they would arrive at two different answers. They might waste time trying to understand what was going on, and it could throw them off their stride.
Furthermore, the answer to part (a) feeds into part (b), so this student would be feeling very uncomfortable with the dilemma as to which answer was correct.
Not all students would be affected. Some might not realise the implications of being given four measurements instead of three, and would proceed to find the answer to the first part, and not do any further checks. They could then proceed to use their answer in solving part (b).
Note 1: the Irish language version of the paper did not include one of the measurements, namely the 36 degrees angle at H. Because of this, their question 8 was legitimate. However, this ‘omission’ was noticed, and ‘corrected’ verbally by means of an announcement by examiners before the exam began. This meant that Irish language candidates then became embroiled in the same problematic scenario.
Note 2: There’s a second error in the question, in that an exterior angle is effectively described in the text description as being 124 degrees, but it is the interior angle which is labelled as such in the diagram. It is highly likely that most students would have gone with the diagrammatic description, but it’s still not acceptable to have this kind of error, especially as different students have different ways of processing information. This could have been another source of confusion.
There are problems with all of these approaches. Too ‘soft’ an approach is unfair to students who are genuinely strong on trigonometry, as it effectively levels everyone’s marks for this question. Too ‘hard’ an approach could penalise all students unfairly, including the very best students as mentioned in scenario (c).
None of these approaches can compensate the students who spent extra time on this question, or who lost momentum or confidence in the exam due to this question.
The SEC has stated that the official marking schemes will be available by the end of August, in time for students to view their scripts. However, they may release a statement before then, due to the notoriety of this particular incident.
Anecdotally, there has been a mixed reaction: Most students, parents and teachers have been rightfully upset that this question could be set with these significant errors, not to mention other errors seen in other exam papers, including maths, this year. A lot of students have said that they hadn’t realised that there was an error in this trigonometry question, and as such were unaffected on the day. However, they are still concerned about how the question will be marked.
Some other students experienced unease at the question having too much information, but in the exam situation, took it at face value and did not necessarily investigate too deeply. Others did the check and became confused when they got two different answers. This caused a delay and upset during their exam.
How can this be prevented from happening again?
Exams are drafted and set by a team, including at least one drafter and one setter, and possibly assistant setters. These are answerable to a chief examiner who has ultimate responsibility for setting the exam. The setting process includes proof-reading, review and revision in accordance with SEC procedures.
Human error is always possible in any endeavour. Although the exams are set by a team of subject matter experts, it’s important that the quality assurance process be well designed, and fully complied with.
It is difficult to understand how so many errors have slipped through this year, particularly in maths. To avoid this in future, there should be more clarity and accountability in the process.
A report is being prepared for the Minister regarding the mistakes made in setting exams this year. We hope that there will be no more such mistakes, particularly in maths, from 2014 onwards. Irish students must be able to trust that their exams are based on valid and fair questions ongoing.
Eamonn Toland is founder and Managing Director of TheMathsTutor.ie, and leads a team of mathematicians, teachers and technologists who have created Ireland’s leading e-learning system for Project Maths.
To embed this post, copy the code below on your site
have your say