Mary Beard emailed me in response to my little rant yesterday (I could blame the late hour but what's my excuse at other times?) on A-levels pointing me to her very interesting article in The Times Literary Supplement which is well worth a read.
This paragraph struck home for me:
"I know of at least one A level examiner who has given up because he was forced to mark down candidates who wrote really intelligently about a subject but didn’t give the points that were demanded by his ‘marking criteria’.
I never did A-levels, leaving school as I did at 16 (I later did my degree and MA with the Open University while working full time - these kids today don't know the meaning of the word 'stress'!) but I recently found my old O level and 16+ certificates which have just passed their 20th anniversary. I failed English Literature, Religion and Art. (E, E, and D respectively). I got an A in English Language (which is writing, effectively) but failed the exam on comprehension of, in my case, Jane Eyre and The Importance of Being Ernest, two texts which I now count as among my favourites. My problem, thinking back, was that I was a bit too clever for my own good, taking issue in my essay on Jane Eyre, with the idea of literary criticism that pretends to know what was in the author's mind. A bit precocious, no doubt, and certainly not answering the question, but I wonder if the examiner who failed me didn't think my answer was worth a bit of discretion.
Religion I failed because I was going through my rebellious phase at the time and remember distinctly challenging the Catholic church's policy on abortion and, for some reason, nuclear disarmament.
Art I failed because, according to the teacher who marked me (it was a moderated course work exam) I 'lacked confidence'. Way to boost that, miss!
I wonder if she knows what I do for a living now?
Anyway, the issue of rewarding risk and unexpected outcomes is one that always comes up in art and design education whenever assessment criteria and learning outcomes are discussed (of which more shortly) but the easy answer to that is to write that into the things, but at least make it transparent to everyone concerned.
This next section in Mary's article also chimes with me
"When they get to university the hang-over of this is still horribly apparent. Students will press you to say what kind of class you think their essay would be given. If you respond ‘a 2.1’, their next question is likely to be, ‘So what have I left out that would get me a first’. As if getting a first was simply about fulfilling all the assessment criteria.
But tub-thumping about standards is a bit of a thoughtless response to all this. The sad thing is that the tick-box style of marking is an almost inevitable consequence of the very proper attempt to democratize A levels. It’s all very well thinking that the open ended intellectual essay style is what should be rewarded. But what do you do if you go to a school where they don’t know the rules for that genre? Isn’t it reasonable for you to expect to be told what you would need to do to get an A?
Perhaps even more pressing is the question of the examiners themselves. In the old days, when A levels were a minority option, you had a small group of experienced (and, no doubt, underpaid but devoted) examiners. You might trust them to make reasonably independent judgments about a kid’s essay (and, in any case, the numbers were small enough for them to be checked up on). Our recent mad fixation with formal assessment has more than quadrupled the numbers of examiners that are needed – the demand being such that in some subject trainee teachers are used to mark the most important tests in a child’s career. So, of course, we have to generate firm rules and fixed criteria, simply to train and police the examiners.
The real question isn’t whether we are dumbing down. It’s what on earth we think all this examining is for. If it’s for choosing the brightest, it’s a blunt, time-consuming and inefficient instrument indeed. But maybe that’s not its point – and we should be thinking of quite different ways to do that."
(A quick aside... Several years ago, when the course I ran was validated by Oxford Brookes, I was told never to use a classification (2.1, 2.2 etc) as a mark, only as a degree, and I've stuck to that advice ever since. But it's still quite common to hear first year students talking about getting a 'first' - to me that sends the wrong signal, that simply maintaining that level of performance will guarantee a first at graduation)
Now I'm probably going to go off at a slight tangent here. (It's part of my charm).
I'm a big fan of learning outcomes and assessment criteria, things that are fashionably unfashionable at the moment, mainly because very few people can write the damn things properly and so assume the idea is at fault rather than the implementation. I think Mary's critique is spot on and she identifies the same issue: that we are using the wrong methods for the things we are trying to identify, and that these methods are directing students' learning.
John Biggs, in "Teaching for Quality Learning at University: What the Student Does", describes exactly how this happens and what to do about it. It's a must-read for anyone involved in assessment and apart from anything else makes you realise that assessment is teaching, in that students will quite rightly (as do we all) work out what has to be learnt and done based on what it takes to pass. Writing your course and its assessment in a way that encourages 'deep learning' is the key to avoiding the issues Mary identifies - but that is more than, as some would have it, simply telling students that to get an A they have to read lots of books and write lots of essays. Learning is a strategic activity - it's a basic human characteristic - and as much as it might pain the more romantic (read dinosaur-like) academics to acknowledge it, so is teaching.