Module 13

Approaches to Assessment

This module is divided into the following sections:
 
Outcomes

Author of this module

13.0 Introduction

13.1 General issues in assessment
13.1.1  What do we mean by assessment?
13.1.2  Why do we assess?
	13.1.3  Learning outcomes
13.1.4  Key features of assessment
13.1.4.1	Validity
13.1.4.2	Reliability
13.1.4.3	Transparency
13.1.4.4	Practicality
13.1.4.5	Washback on teaching
13.1.5  Formative and summative assessment 
13.1.6	What do we do with our marks?
13.1.6.1	Norm-referencing
13.1.6.2	Criterion-referencing
13.1.6.3	A mixed system? 
	13.1.7  Reducing the marking burden
13.1.8  Accredited courses
	13.1.9  Tackling plagiarism

13.2  Examinations
13.2.1	Exams and continuous assessment
13.2.2	Open-book exams
13.2.3	Setting exams
13.2.4	Moderating exam papers
13.2.5	Marking exams
13.2.6	Double-marking

13.3  Alternative forms of assessment 
13.3.1	Self-assessment
13.3.2	Peer assessment
13.3.3	Group assessment
13.3.4  Computer-aided assessment

13.4  Understanding language errors
	13.4.1  Cause for despair?
	13.4.2  Errors and second language acquisition
	13.4.3  Why do students make mistakes?
	13.4.4  Responding to errors

13.5  Feedback and error correction
13.5.1  Providing feedback
13.5.2	 Correcting errors in written work
13.5.2.1	Feedback and ICT
13.5.3  Correcting spoken errors 
13.5.4  Self-correction

13.6  Summary

13.7  Commentary on activities

13.8  Appendices
Appendix 1 	Sample feedback sheet (written work)
	Appendix 2	(a) Criteria for assessing long essay 
(b) Criteria for assessing student video presentations
Appendix 3	(a) Open University criteria for speaking tasks
(b) Open University criteria for written work
Appendix 4	Error classification sheet for German

13.9  References

13.10  Recommended further reading

13.11  Assessment task

Outcomes
By the end of this module you should be able to:
•	formulate effective learning outcomes for any language module you teach;
•	show understanding of the key features of assessment and devise assessment tasks accordingly;
•	use formative and summative assessment appropriately in your teaching;
•	apply criterion-referenced marking schemes to all your assessment tasks;
•	identify ways to reduce your marking load;
•	explain the relative strengths and weaknesses of exams and continuous assessment;
•	adopt sound and fair procedures in marking and moderating;
•	employ a range of non-traditional forms of assessment;
•	appreciate why students make errors in language work;
•	respond appropriately to students’ errors and provide effective feedback.

Author of this module
John Klapper is Director of the Centre for Modern Languages and Professor of Foreign Language Pedagogy, University of Birmingham. He is a trained teacher, a member of the ILT, a HEFCE National Teaching Fellow and from 1997–2001 co-directed the FDTL-funded DOPLA project. His current research interests are language teacher education, factors predictive of success in foreign language learning and the development of undergraduate language competence. He has published on different aspects of the learning and teaching of foreign languages, including teaching methodology, content-based learning, L2 reading, and has produced various commercial materials for the teaching and learning of German and Russian. 

Contact: j.m.klapper@bham.ac.uk

13.0  Introduction

A bit like teaching itself, assessment is a topic that suffers from the fact that everyone has had first-hand experience of the process (as ‘victim’, so to speak) and has therefore certain fixed views on it. We have all come through it, more or less unscathed, and this can make us reluctant to stop and reflect on whether we knew enough about what was going on when we were being assessed, whether the process we underwent was a fair one, and whether the assessment measures employed were appropriate. This is the principal reason why some of the methods of assessment in use today in higher education (HE) have not changed substantially despite major developments in approaches to other areas of language learning and teaching. One of the aims of this module is to get you to consider how you currently assess, to reflect on why you do it this way and to look at alternative approaches.

Activity 1

In Module 1 you were asked to elaborate a set of personal beliefs about teaching and learning. As a preliminary to our work on assessment, refer back to this statement of beliefs and produce a similar statement of your views on assessment and feedback. To help you on your way, have a look at the following two statements about assessing language work that I asked two colleagues to produce. Both were asked to list a maximum of 10 beliefs and to limit each statement to 25 words.

Tutor 1 works in a department of languages and teaches primarily ‘specialist’ students of modern languages; Tutor 2 works in a language centre and teaches on an institution-wide language programme. 

‘What I believe is important in language assessment’

Tutor 1
•	Students need regular homeworks; they need to do weekly written work.
•	Course work should be assessed but should count a little less than end-of-year exams.
•	Assessment is about being formative as well as summative; course work should help students to learn better and feedback is crucial to this.
•	Translation is the ultimate test: it pins students down, they cannot hide their lack of knowledge, like they can in comprehension.
•	English > FL translation is something for final-year work only.
•	Vocabulary needs regular testing. Otherwise students won’t learn it.
•	Students need the chance to redeem any failures.
•	It’s probably not really a belief, but we must assess efficiently. We have so little time these days and lots of students to assess.
•	Both exams and course work should be challenging. They’re meant to examine what students know, not make them feel good.
•	You cannot separate language and content. Students need to learn they are here not just to learn the language. Assessment criteria need to reflect this.

Tutor 2
•	It is important to assess all four language skills, plus knowledge of grammar.
•	Oral work should, on the whole, take precedence over written work.
•	Generally, we are not professional enough in our approach to assessment. University language teachers ought to be trained in assessment.
•	Students should be involved in assessing their own work where possible.
•	Assessment tasks should be as naturalistic as possible, ie tests should be open-book and allow use of dictionaries.
•	Translation into the foreign language is an inappropriate task for students.
•	Assessment must be equitable: some students panic in exams, while others like them. Balancing assessment methods is the only way to be fair to all.
•	We should use the full mark scale from 0–100. Even the best student rarely gets more than 75 or 80 in our marking culture.
•	Transparency is very important. Students need to know why they are being assessed and exactly how marks are being allocated.
•	Teaching and assessment need to be dovetailed. We should never assess anything that hasn’t been taught.

Now it is your turn. Same rules apply: 10 beliefs, maximum 25 words each. Remember the above examples are for illustration only and should not constrain your own reflections which can range over the whole field of language assessment. Keep a written record of your beliefs as you will need them again at the end of the module.

Besides encouraging you at various points to reflect on your own approaches to assessment, this module provides an overview of important general issues in assessment. It considers the examining process and alternative approaches to assessment, helps you to understand language errors and looks at ways to provide feedback on assessed work. At certain points in the module you will find extracts from interviews with  language tutors and students on the subject of assessment, entitled ‘Tutor voices’ and ‘Student voices’ respectively. 
This module is not concerned with specific forms of language testing, for which you are referred to Module 14. Ideally these two modules should be worked through consecutively, but in any case you will find a number of cross-references between them.

13.1  General issues in assessment
In this section we look at a number of generic issues that are central to the business of assessment. After considering the nature and purpose of assessment and the key role of learning outcomes, we contrast formative and summative assessment and discuss the different uses assessment can be put to. After suggesting some ways in which you can lighten your marking load, we explore student resistance to assessment and how it can be overcome, as well as the vexed question of plagiarism.

13.1.1  What do we mean by assessment?
The word ‘assessment’ is usually used to refer to the activity of judging, commenting on and awarding a specific mark to a piece of work. As we shall see, this activity can take a wide variety of forms. There is another sense of the word, however, a broader one which denotes the monitoring of performance that a tutor is constantly involved in. This monitoring is most frequently informal, the fruit of careful listening and observation of student performance, and is used to help inform subsequent teaching decisions, either at the micro-level (‘Shall I adjust the next example to reinforce this point as he/she seems uncertain about these lexical items?’) or at the level of a whole class (‘Tomorrow I’ll move on to the next section of the book as they seemed to cope really well with this dialogue today’). Both forms of assessment are important constituents of the pedagogical process but this module will be concerned primarily with the first of them.

13.1.2  Why do we assess?
We may choose to assess students for a variety of reasons. The main ones have traditionally been to:
•	grade or rank students;
•	compare students’ performance relative to each other;
•	pass or fail students;
•	facilitate student progression, in particular to ensure a prerequisite level of ability for further study;
•	contribute to final degree classification.
This view of assessment has been broadened in recent years to encompass more diverse, teacher- and learner-orientated conceptions. Assessment is seen as something that should arise as naturally as possible out of the teaching process, rather than as something separate that is ‘bolted on’ to the end of a module. Far from merely providing a summative, pass-or-fail judgement at the end of the course, assessment is now generally accepted as a central feature of all learning and teaching which not only provides important feedback to teacher and student, but also, in the process, helps to shape subsequent learning and teaching. 
From the tutor’s point of view, assessment provides information on the following:
•	Have the learning outcomes been met?
•	How effective has my teaching been?
•	How effective have the learning materials been?
•	How have particular individuals performed? Do I need to do more to address their needs?
•	What needs to happen next?
•	How can I improve the course?
From the student’s point of view, on the other hand, the purpose of assessment seems to reside in the following:
•	lending direction and purpose to my learning;
•	providing (positive) feedback on what I can do well and what I know;
•	acknowledging and recognizing my success and thus maintaining my motivation and encouraging further success;
•	diagnosing my faults, areas of weakness and lack of knowledge;
•	providing constructive feedback on how I can address my shortcomings  and improve my performance.
On a less idealized, more pragmatic view, assessment, especially in today’s market-led HE, is often the principal focus of students’ attention, determining what they see as important and how they spend their study time (Brown et al, 1997). Indeed it is assessment considerations that frequently drive students and dictate their choice of options and learning tasks: 

[…] assessment systems dominate what students are oriented towards in their learning. Even where lecturers say that they want students to be creative and thoughtful, students often recognise that what is really necessary, or at least what is sufficient, is to memorise. (Gibbs, 1992: 10 )

Such a stance governs the behaviour of those students who learn ‘strategically’, ie those who realize that they need to learn only certain things to obtain a reasonable mark in an assignment or exam, and will therefore ignore or avoid what we consider desirable additional or alternative learning activities. This has clear implications for us as tutors. We need to recognize that, despite what we may think should motivate students, in practice if we want  to change the way students learn, then we need to change the way we assess them. See also section 13.1.5.) 

13.1.3  Learning outcomes
The new approach to Academic Review by the Quality Assurance Agency is characterized by an emphasis on the notion of ‘alignment’. This refers to the process of bringing into line and linking to each other the key phenomena of aims, intended learning outcomes, methods of learning, assessment methods and tasks, criteria, marking and feedback to students (see Brown, 2001: 4–6). Biggs (1999) talks about ‘constructive alignment’, by which he means that learning is the result of constructive activity on the part of the student and that effective teaching aligns teaching methods and assessment procedures to the learning activity specified in the course objectives; all parts of the process should serve to support appropriate learning (Biggs, 1999: 11). The present module looks at this interrelationship in the field of language learning and teaching, but this section focuses specifically on the question of learning outcomes.
Although as linguists, we know that language acquisition is a highly complex process and that no learner ever owes all or even most of his or her language knowledge and proficiency to what goes on in the classroom, it is  as important in language learning as it is any other discipline for manageable learning outcomes to be linked to appropriate methods of assessment so that students, colleagues and quality monitors can be clear about and have confidence in the standard and coherence of our modules. 
Learning outcomes denote the changes that have taken place in the student as a result of a particular learning process or experience. When writing learning outcomes it is a good idea to think in terms of what you hope students will be able to do once they have completed the course. The learning outcomes should serve as your outline for assessment, and with good ones you should be able to answer with ease the question ‘How will I assess this?’ If the answer is difficult to find, this may well mean you need to rethink the outcome and link it to specific, measurable evidence of competence. It is also important that all outcomes you include should be assessed in some way; they need not necessarily all be measured to the same degree, but if it is a significant element of the module, it deserves to be assessed. (See also sections 13.1.4.1 on validity and 13.1.4.5 on washback.).
Consider  the following illustration, taken from a beginners’ Spanish course aimed at non-specialist linguists:

On completion of this module you will be able to:

•	speak with a degree of fluency about yourself, your life, interests and course of study;
•	read and understand simple short articles in Spanish aimed at learners of Spanish as a foreign language;
•	listen to and understand in detail selected short examples of Spanish spoken at a natural speed and understand the gist of more extended speech;
•	write letters and short essays in Spanish on a limited range of predefined topics; 
•	demonstrate knowledge of and the ability to apply in context the main aspects of basic Spanish grammar (see detailed grammar outcomes);
•	show understanding of a specified body of vocabulary appropriate to the topics covered (see accompanying vocabulary lists);
•	use relevant reference materials; 
•	work independently of the teacher;
•	show awareness of basic aspects of Spanish-speaking cultures, including working life, health issues, food, travel, important dates and key festivals.

These general outcomes (or ‘objectives’, as some institutions call them)  are acceptable as far as they go, but they are not sufficient. There should also be a clear link between the outcomes that we propose to assess and the specific language (grammar, vocabulary, idiom) that students will require command of in order to demonstrate those outcomes. This does not always happen in HE at present. As Schmidt and O’Dochartaigh (2001: 43) note in their research into HE modern language programmes:

Within language-based modules there is frequently no attempt to establish any kind of thematic unity where vocabulary acquisition is concerned. Staff teaching different aspects of a module will often choose texts dealing with wildly different topics on a weekly basis, while someone teaching translation or essay-writing within a module may have no idea what topics are being discussed by the students in conversation classes. Thus students hop from pillar to post on a daily, even hourly basis, and the target language seems like a never-ending mass of vocabulary that they will never really succeed in mastering […]. There is a broad assumption in many degrees that by the end of a degree course a student ‘knows the language’, and therefore an exam based on any aspect of language can be thrown at the students. The assumption is probably unfair in all but the fewest cases.

Some may dispute this final point, arguing that a final exam in a modern language degree should indeed aspire to at least something approaching such a comprehensive assessment of linguistic ability. However, overall the argument is a sound one and the point well made that language courses and their various components need to be co-ordinated and their taught linguistic content linked as far as possible (it can never be entirely so) to clear learning outcomes. A truly coherent language degree programme would include blocks of lexical content, say four per year, relating to the type of general outcomes above, which would feature in coursework and in end-of-year assessments. This need not limit tutors to only this vocabulary but it would serve as indicative content, acting as a focus for students’ learning and a framework for tutors’ assessment.
	In the same way, learning outcomes for grammar should be formulated for each phase of the programme, especially for the early stages of language learning. This is, in effect, what happens on ab initio or post-GCSE courses  that employ textbooks with clearly structured grammatical content. Experience suggests it is unlikely to happen on post-‘A’ level language modules. Yet students have a right to know what knowledge and (broad) lexical areas they are expected to cover at each stage of their course. This need not limit able students who will, in any case, read and listen widely beyond the minimum requirements to extend their proficiency. But it will provide crucial guidance to other students on the anticipated learning outcomes of particular modules and will help the tutor to devise more appropriate assessment tasks.

13.1.4  Key features of assessment

Activity 2

Before working on this section, note down four problems that can affect the success of language assessments:

  _____________________________________________________
 
  _____________________________________________________

  _____________________________________________________

  _______________________________________________________

We all experience occasions when the assessments we set ‘don’t work’ very well. This may be because we find that what we have designed fails to assess what we intended it to. Alternatively, the tests may turn out to be too easy or too hard. In marking, the mark scheme may prove to be too inflexible or too permissive, or we may find we are not treating all candidates equally for some reason. When marking and returning work, it may become evident that students were not clear about what it was they were being assessed on. Or we may find the assessment activities prove unduly cumbersome or time-consuming. Finally, over a period of time, we may notice that the type of test we are setting is affecting the way students learn or the way we teach during the course, eg both we and they may just be teaching and learning ‘for the test’.
	These effects are common in varying degrees to all attempts to assess students and they can be summarized under the following headings:
•	Validity
•	Reliability
•	Transparency
•	Practicality
•	Washback on teaching
We shall briefly consider each of them in turn.

13.1.4.1  Validity
There are various aspects to validity, including:
•	‘face validity’ (does the test look as if it measures what it is meant to?);
•	‘content validity’ (does it contain a representative sample of the skills and knowledge it is meant to be assessing?);
•	‘criterion-related validity’ (to what extent do test results agree with those derived from an independent and dependable measure of a student’s ability?);
•	‘construct validity’ (can it be shown that the test measures the ability it is supposed to measure?). 
The latter two aspects are research procedures which are most relevant to the production of mass standardized tests (for further details, see Hughes, 1989: 23–27). For university language teaching purposes, the first two are more important. 
	Face validity is important because tests that are not thought to be relevant to the supposed purpose of assessment may not be accepted by students and this may reflect how they perform on them. For example, cloze and C-tests (see Module 14, section 14.5.1) may not be considered meaningful in assessing the ability to write the foreign language (FL), or a writing test may not be thought relevant to a reading knowledge course. Content validity is even more crucial: if a test ignores whole areas of a syllabus this will have a very negative effect on teaching and learning (see ‘Washback’ below, 13.1.4.5).  As already noted (see 13.1.3), our assessment needs to be closely tied to learning outcomes. It is essentially a two-stage process in which, first, we identify the outcomes (knowledge or skills) we want to assess, and then decide how it is we are going to test them. Starting from this premise, we are more likely to measure what we want, that is to make our assessment valid. 

13.1.4.2  Reliability
Assessment also needs to test students in a consistent way, so that the first student to be assessed is treated in the same way as the last. When marking 60 pieces of writing in the FL or conducting 60 oral exams, we need to be sure we are applying the same standards across all students, that fatigue or boredom are not distorting our perceptions, that we do not become stricter or more lenient as we work our way through the group of students, and that any marking scheme is adhered to closely throughout. That is, we must ensure our assessment is reliable.
	The ways to achieve test reliability are:
•	Sample widely: the more items included in a test, the more reliable it is likely to be.
•	Restrict student choice: the more candidates can choose from a range of questions (eg in essay writing), the more varied the responses will be across the cohort (thus making marking reliability more difficult to achieve) and the greater the likelihood of a candidate’s performance varying depending on when he/she takes the test.
•	Write unambiguous items: this is best achieved by getting colleagues to check them or by pretesting them in some way.
•	Provide clear, explicit instructions: again, scrutiny by colleagues at draft stage is the best way to avoid problems.
•	Ensure students are familiar with the test format: students will not perform as well if the testing techniques are new to them. Past papers and practice questions are thus important for reliable testing. 
•	Provide uniform test conditions: test outcomes will differ unless conditions (timings, acoustics, support materials) are the same on each iteration of the test.
•	Use as much objective marking as possible (see Module 14, section 14.2.3, on marking written work). 
•	 For tasks requiring subjective judgement (eg orals, essays), use clear and explicit criteria.
•	For more objective tasks (eg most listening and reading comprehensions), use a detailed marking scheme and ensure all markers adhere strictly to it.
•	Train markers: markers should meet to discuss acceptable answers and  sample essays, etc., at the start of the marking process; this is especially important where several markers are involved.
•	Mark anonymously to avoid any unwitting bias. Studies suggest even having the name of an unknown student on a script can affect the marks awarded.
•	Employ double marking, ideally of the blind variety (see 13.2.6).
(Adapted from Hughes, 1989: 36–43.)

13.1.4.3  Transparency
Clearly, in any assessment task it is crucial that students know what they have to do. But it is just as important that they know what the marker is looking for, what aspects of the task or what sub-skills of the activity they will be assessed on. It is no good having assessment procedures that bear little relation to the stated learning outcomes of a module. And it is no help to students to be told that in a language test they will be assessed on their knowledge of French or Spanish: they need to know which aspects or which sub-skills are to be assessed (eg in writing: content, organization of material, accuracy, range of language, etc). Similarly, students need to know by which standards these things will be assessed, ie they need to see and understand the marking criteria. All these issues are aspects of transparency in assessment; besides making the process transparent to students, they are also central to demonstrating to external reviewers the coherence and quality of assessment procedures.

13.1.4.4  Practicality
We all work in less than ideal circumstances for one reason or another, and it is not always possible to have what we would consider optimum conditions for assessment. In other words, we operate within certain practical constraints. One of the most common of these is time restriction: ideally we might like to trial our tests and exams before letting students loose on them, and we might then like to remove less reliable items from the test and adjust our mark schemes to make them tighter. If you are teaching several different modules, however, each with just 15 or 20 students on it, you are simply not going to have time to do this. Such approaches are only feasible and cost-effective in mass assessment programmes, such as those run by the national awarding bodies for GCSE, ‘AS’ or ‘A2’ levels, or by The Open University. Similarly, timetable and resource constraints may well mean it is not possible for all students on a module to be assessed via individual oral presentations twice a year, and group assessments may have to be used instead. In other words, whatever assessment instrument we use also has to be practical.

13.1.4.5  Washback on teaching
Finally, we return to the question of assessment’s links with learning outcomes. We must ensure that what we test is not only relevant to what we have taught and what students have learnt, but also that the assessment process serves to promote that learning. When assessment has beneficial effects on teaching and learning, it is common to talk about ‘positive washback’, whereas if assessment fails to further the learning outcomes, one would see this as ‘negative washback’ (Alderson and Wall, 1993). 
For example, if the final exam in a course on developing reading knowledge in Italian involved students in a role play of a job interview in an Italian firm, preparation for this specific exam task would do little to develop students’ capacity to access articles and reference works in Italian – presumably the main aim of the course. Similarly, if an IWLP course promoting students’ communicative ability in German were assessed purely via translation of a newspaper article on current affairs, students’ efforts to learn relevant vocabulary would do little to help their functional use of German. If, on the other hand, in a course on the French political system, text and oral work in French led to student oral presentations on the pros and cons of the parliamentary system, then the assessment task would dovetail with classroom work, would provide a logical connection with and would follow on naturally from the process of learning and teaching, while serving to motivate student learning towards a clearly identified, relevant and attainable goal. This is an example of positive washback. (See also section 13.1.3 on learning outcomes.) 
To summarize, the key things to consider in ensuring positive washback are:
•	Test the knowledge and skills you want to encourage, not just those that are easiest to assess.
•	Test the learning outcomes directly: if we teach students to write summaries, then they must be tested on that skill.
•	Make sure students know what is expected of them in the exam or test.
•	Test a representative sample of the syllabus by choosing items from a wide range of topics, rather than just one narrow area.

The holy grail of assessment is for all of these features to coincide, for assessment to be valid, reliable, transparent, practical and to promote positive washback. Sadly, it is rarely possible to satisfy all these demands simultaneously. Objectively scored placement tests (see Module 14, section 14.1.1.2) that assess students’ writing and speaking proficiency and their command of vocabulary are likely to be valid, transparent and reliable but will almost certainly prove too time-consuming in the very hectic first days of the academic year. At best, we might give students a multiple-choice grammar test to provide a rough idea of which group we should allocate students to. Similarly, tests that promote positive washback, like the French oral presentations mentioned above, may prove to be unreliable and to lack transparency if students are given different sub-topics of variable difficulty or accessibility (eg the role of parliamentary committees or the electoral system might prove for whatever reason to be more challenging than discussion of the role of the president). In practice, compromise is likely to characterize all our endeavours to reconcile the five key features of assessment, but we must nevertheless always remain aware of them in devising tests and exams.

13.1.5  Formative and summative assessment
One of the most useful and instructive things we can do as language tutors is occasionally to stop and ask ourselves: Why am I getting students to do this? What is the purpose of this assessment? Two more specific formulations of these questions are: 
•	Am I using it to measure the stage students have reached on their way to achieving the particular course outcome, that is: am I more concerned with the product? 
•	Or am I using it to help students improve their performance in the language, that is: am I more concerned with the process?
These two purposes are usually called summative and formative assessment respectively.
	In formative assessment, what matters is the dialogue between tutor and student: the amount, quality and appropriateness of feedback and students’ response to it are more important than the marks awarded. Summative assessment, on the other hand, is an overall judgement of attainment at a particular moment, a clear and supposedly definitive statement of performance rather than a contribution to the teaching and learning process. This neat distinction does not always work in practice as the two types of assessment can often overlap, especially if feedback is given on an assessed class test or marks are awarded for all the work set on a module. (For a readable summary of the key differences between formative and summative procedures in a generic rather than languages-specific context, see Knight, 2001: 3–10.)
	Ask any tutor and they will tell you that ideally they would like their assessment to serve more the interests of formative than summative assessment. However, they will also quickly add that the latter is an essential requirement of our educational system, indeed of our society, that constructive feedback and encouragement are inevitably accompanied by marks, exams, credits and certificates. One can argue about whether summative assessment is the most effective form (it is almost certainly not), but society demands it and (depressingly, in the eyes of some) students see it as the main purpose of assessment (see section 13.1.2); tutors therefore must employ it.
	That does not mean we should not be trying to educate students about assessment, as well as about our disciplines. Indeed it is our duty to show students the importance of assessment in their studies as whole, not just as an exit token or stamp. We need to demonstrate that assessment and feedback are central to making progress in a language, that they help prepare the way for further effective learning and progression, and that they underpin motivation, which research has clearly shown is the key factor in learning, in particular learning a language.
	And, of course, assessment also brings rewards. However much the purist in us values formative assessment, our pragmatic side must acknowledge that students are motivated by the prospect of success and certification, and that we would be foolish not to harness this motivation as much as we can: the reality is, if you tell students they will be assessed on something, they are likely to spend more time learning, digesting and assimilating it than would otherwise be the case.
	In summary, assessment is inevitably both formative and summative. There is nothing wrong with the fact that assessment drives learning provided it does not merely become a means of sorting the ‘wheat from the chaff’ and that we do not just test students for the sake of it. The key is to start from what one wants students to learn, the outcomes, and to devise forms of assessment that help them to realize those outcomes.
 
13.1.6  What do we do with our marks?
A crucial distinction in the world of assessment, relating to how we interpret the results we obtain from our exams, tests and homeworks, is that of norm-referencing and criterion-referencing. 

13.1.6.1  Norm-referencing
Norm-referenced assessments are those in which result boundaries are not fixed but are determined by a comparison of how students in a particular cohort have performed relative to each other. For example, it might be decided that in a given exam the top 10% of the resultant marks are to be classified as ‘A’, 30% as ‘B’, 35% as ‘C’, 20% as ‘D’ and 5% as ‘E’. Just by knowing that a student scored, say, 82%, we would not be able to tell whether he/she had got an ‘A’, ’B’, or whatever. It is only the overall ranking, ie the student’s performance relative to the rest of the cohort, that would give us this information. This was the system employed in UK public exams several years ago before the advent of GCSEs, and it is still used in those areas of the country that employ 11+ or school entrance exams: there are a certain number of places available and therefore only the top x% will be successful and the minimum mark required to ensure success will vary from year to year. Generally speaking, other than in such highly competitive circumstances where places are strictly limited, norm-referencing is now both less common and less desirable.

13.1.6.2  Criterion-referencing
Criterion-referenced assessment, on the other hand, involves awarding marks in relation to a predefined objective or criterion, independently of how all the other test candidates have performed. The most common example quoted is the driving test where everyone needs to achieve a minimum threshold of competence, otherwise they will not be allowed to drive on their own. And this threshold remains the same regardless of how many pass or fail on a single day or in a single year. Performance is measured purely against certain clearly specified criteria, not against other novice drivers’ performance. In language testing, an example of a criterion for a basic pass mark in an oral exam might be: 

Can communicate most of the intended message with some hesitation and occasional lack of clarity. The performance features numerous formal errors and is characterized by a limited range of vocabulary. Pronunciation and intonation are likely to be poor but accessible to a sympathetic native speaker. 

Examples of assessment criteria for two different types of language task can be found in Appendix 2 and more composite criteria grids for speaking and writing from the Open University’s language programmes are listed in Appendix 3. 
	From the student perspective, one of the key differences between norm- and criterion-referencing is that in the former it is impossible to know what mark one needs to achieve in order to obtain a particular grade; in the latter, by contrast, not only is this information vouchsafed but students are given a description of those attributes the marker is looking for at each grade. From the tutor’s point of view, a major difference is that in a norm-referenced test, in which an ‘A’ is awarded to the top 10% only, an exam set at too high or low a standard is less of a problem, since grades are allocated purely on a relative basis: if the top 10% of marks one year are all over 80, whereas no one scores over 75 the next, this is not especially important. In a criterion-referenced system more care has to be taken both in setting the paper and in allocating marks strictly in accordance with the agreed criteria. There is less room for exams of variable quality in a true criterion-referenced system, which is why so much time has to be invested in producing successive drafts of each paper by awarding bodies as they strive to ensure public exams, such as GCSEs, are pitched at the right level each year. (For further discussion on the subject, see Knight, 2001: 16–19.) 

13.1.6.3  A mixed system?
There are a number of ways in which this norm/criterion distinction is not as clear-cut as the above might suggest. First, even in such mass systems of essentially criterion-referenced assessment as school awarding bodies or The Open University language programmes, norm-referencing still plays a role in ensuring consistency from year to year; ie it is expected that, broadly speaking, a comparable number of students will fall within the major grade boundaries. This acts as a check on standards and on the level of difficulty of the exam, which may have proved more or less challenging in a particular year. (Yet the gradual inflation of GCSE and ‘A’ level results over recent years shows that the system still allows for year-on-year improvements in performance.) 
Second, criterion-referencing almost always involves a measure of ranking too. In universities, for example, most categories described by criteria are broad ones (eg 60–69) and, consequently, once a student’s performance has been allocated to one such category, the marker still has to decide where within the category to place the student (is it 62, 65 or 68?). Usually the only reliable way to do this is to compare the student’s performance  with that of another student in that band – eg ‘I gave Smith 62 but this was a bit better, therefore 65’. (It may be questioned whether this is meaningful or reliable, and therefore whether it is a sensible use of staff time both in marking and subsequent moderation, but the fact is that most mark systems require it and so you are likely to have to do it.)  
	Universities nowadays have very full regulations concerning the allocation of marks on modules and the profiling of degree classifications, and most module handbooks include criteria grids. Alongside this apparent universal acceptance of criterion-referencing, however, one can detect a sort of informal parallel norm-referencing. To illustrate this, try awarding eight first-class marks in a group of 15 students and the chances are that aspersions will be cast by colleagues on (a) the quality of your assessment task, and (b) the soundness of your marking. This is because most tutors retain, alongside their commitment to criterion-referencing, a notional norm or quota of students that should be awarded a first-class mark, a 2.1, etc. There is no doubt that this cuts across a commitment to criterion-referenced assessment, and that  limiting marks to, in effect, the relatively narrow 35–75  band fails to adequately distinguish student performances and thus serves to distort results. However, given the current HE assessment culture, this practice does act as a check on grade inflation: if individual markers are awarding too many  marks over 70%, this distorts students’ final profile or mark average and leads, ultimately, to larger numbers of students being awarded first-class degrees. Conversely, an excessive run of low marks can lead to a higher proportion of failures. University exam boards, in collaboration with external examiners, are charged with ensuring that quality and standards are broadly maintained from one year to the next, but it is at the level of the individual tutor marking a group’s assignments that this process starts.
 	These musings lead on to bigger issues, such as how, in a criterion-referenced system in which successive cohorts of students are supposed to be assessed according to the same criteria, the proportion of firsts a department awards can rise over a ten-year period. Or how any university can exhort its departments to award more firsts. Add to this the strongly held views of many academics in modern languages that the linguistic demands made on students and the quality of students’ written work are both lower than they were 20 years ago, and throw in the much wider ability range now appearing in most language programmes, and one might well wonder how one can still talk about maintaining absolute standards in the assessment of language proficiency. (As the earlier reference to ‘rising’ GCSE standards shows, this applies more generally: it has been shown that students throughout education systems are being awarded higher marks; see Elton, 1998.) Of course, one cannot claim any such thing, but no university is going to admit this in public. For whatever reason, standards do change over a period of time, usually very gradually and sometimes imperceptibly, but what is important is that, as far as they can, tutors ensure a consistency of approach in their own work and that, allowing for the quality issues mentioned in the previous paragraph, they seek to apply high-quality criterion-referenced mark schemes. The emphasis on the quality of the mark schemes is important, since good criteria that differentiate effectively between categories  (I, 2i, 2ii, etc) will reduce the need for informal norm-referenced adjustment.

13.1.7  Reducing the marking burden
Languages are notorious for their heavy marking load. Frequent practice is essential and students need feedback on their performance at regular intervals. However, if you are teaching several language modules, the burden can soon become considerable, especially at advanced levels. The following suggestions might help to keep you sane:
•	Ask yourself if every piece of homework is essential: if the module allows it, reduce the number of homework tasks to be handed in and find other ways to provide feedback.
•	Get students to swap and mark each other’s work, especially basic or mechanical grammar exercises. This may smack of primary school practice, but students will readily accept it if you do not do it too often and you ensure everyone has actually done the work (otherwise resentment and cynicism can spread rapidly). Indeed there is growing research evidence that improved learning results from the experience of peer marking (see also section 13.3.2). 
•	With translations that students have done to a standard for you to mark, you could occasionally put a model version on the OHP and work through it with the group.
•	Again with translations, get students to do a first draft; use part of a class to discuss these drafts as a group and then have students do a second draft: the inevitably more accurate second version will be less time-consuming to mark.
•	Instead of setting two essays or two translations and marking them both in the same detailed way, get students to do a first draft; look through this quickly, underlining but not correcting problem areas; then ask students to review their errors and do a second draft which again will be more accurate and therefore cost you less time. This is not just skimping on effort; there are good pedagogical reasons for doing this (see Module 9, section 9.3.4  on writing skills, and Module 11 on translation, in particular section 11.3.2.2.). 
•	Reduce the length of exercises. Do students need to translate a whole passage every time or will, on occasions, written translation of key sentences or paragraphs, allied to thorough preparation of the rest of the text, be sufficient? Or can essay lengths be reduced occasionally?
•	Use tick-box criteria grids (see Appendix 1 and Appendix 2) to allow quick feedback on the most common or typical features of a task, and use detailed comments to address more complex or individualized issues.
•	Consider e-mail feedback: this allows comments you are likely to write on numerous pieces of work to be cut and pasted into messages.
•	With exams, mark each question for the whole cohort in turn (eg all the Question 1 answers first) as this will mean you become familiar with the relevant mark scheme more quickly. It will also mean you are likely to mark that question more consistently.
It is sometimes suggested that using self- and peer assessment are ways to reduce the marking burden. In fact, as sections 13.3.1 and 13.3.2 show, if they are done properly this is rarely the case.
	We should not forget that assessment can also be a burden to students, as the following two extracts illustrate. How would you respond to the concerns of these two students? 

Student voice 1

[1st-year student of Spanish and German]

Those of us doing two languages have so much assessed work for language classes that I start off the semester trying to be really conscientious and spending time doing the essays, summaries and translations as well as I can and making notes on vocab. and so on.…but as more and more exercises are set, it’s just a case of getting them out of the way and moving on to the next one. I’d really like to spend more time on each of them…I reckon I’d get more out of it…but it’s just not possible.

Student voice 2

[2nd-year student of French and Italian]

It’s annoying that lecturers are so obsessed with their own course that they never seem to be aware we have other modules to do. Last semester, for example, I had three major assessed pieces all in the same week. That’s really tough, it’s not fair.

Click on ‘Commentary’ for feedback.

13.1.8  Accredited courses

Student voice 3

[Letter from an Occasional Student] 

I didn’t come here to do a certificate. I’m 68 and am learning French for fun. I don’t want credit and so I see no point in doing your tests.

Nowadays, most language learning in universities does not take place within specialist degrees but in institution-wide language programmes and continuing education-type courses for adult learners. In the past, many among this heterogeneous group of ‘non-specialist’ students attended non-accredited ‘liberal education’ language courses and did so largely for enjoyment and recreation. The advent of large-scale accreditation in adult education, including universities, and the fact that course funding is usually dependent on retaining students and having them complete assessment procedures, has presented language tutors with a real challenge. As the extract above illustrates, students can be resistant to testing or assessment and often do not require any form of certification.
One of the reasons for this is that adults do not have confidence in themselves as language learners and are not keen on anything that might require them to demonstrate their ability or proficiency, especially if they are not especially strong or able students. In such circumstances, tutors have to explain the value of having an assessed component in the course and to show fully and clearly what is required of them in terms of assessment. In particular, they should:
•	emphasize that students will only be assessed on what they have done on the course;
•	make the number and nature of assessment tasks clear;
•	explain the weighting of tasks;
•	talk through the assessment criteria;
•	explain that students are only assessed against these criteria, not other  group members;
•	explain that assessment can help motivate students by providing feedback and giving them a clearer idea of how they are progressing.
From the tutor’s point of view, accreditation also brings clear benefits. For Powell (2001b: 181) it serves as a mechanism to:
•	encourage teachers to be more conscious of learning objectives and outcomes;
•	extend and enhance the value and status of language learning;
•	improve the quality of learning by formalizing good assessment and feedback practice.
While such advice is especially relevant on non-specialist courses, the quality of all students’ learning is likely to benefit from occasional reminders of the purpose of assessment.

13.1.9  Tackling plagiarism

Student voice 4

[2nd-year student of Spanish]

I was pretty fed up last year because someone I know on my course got away with pretty blatant cheating. You bust a gut researching a language essay and getting it done on time and then discover this guy’s done virtually no research, copied most of his stuff off the Web at the last minute and the lecturer doesn’t even notice. Then, to cap it all, he goes and gets a better mark than me!

What can a tutor do about a situation like this? Well, if you do not notice the work has been ‘lifted’ from another source, you cannot do anything. Of course, in language work, the quality of the language is often a real give-away, unless the student has been very clever in integrating it with his or her own work. However, even if you do notice something is not right, it can be very difficult to prove unless you happen to be able to locate the source. With Web material, much work has been undertaken into the development of plagiarism software that can help locate Web sources with relative ease. Unfortunately, most plagiarism is still from books and other printed materials and the source is then much harder to locate. If you suspect it has been copied but cannot locate the source and the student denies any wrongdoing, there is little you can do other than to issue a warning.
	In fact, it is much better to start at the other end. The only effective solution to the problem of plagiarism is to reduce the likelihood of it occurring by being careful about the type of task you set. For example, with coursework essays, do not set too obvious a title, such as ‘Discuss the pros and cons of the EU’, but give it a specific slant – eg ‘In what way does your home town benefit from belonging to the EU? Are there any ways it does not benefit from membership?’ 
	To avoid students getting help with assessed language exercises (eg from international students or others), the best solution is to organize major assessments as open-book exercises under controlled conditions (see section 13.2.2). Here students are allowed access to any reference works they want but are supervised throughout and not allowed to talk to other students. For essays, they might be given the topics in advance and allowed to bring key texts with them. While removing unfairness, this would stop short of formal exam conditions, except for the time constraint. This is indeed a solution suggested by students themselves in Schmidt and O’Dochartaigh’s survey of student views on assessment (2001: 28–29).
The key thing to remember is that cheating is demotivating for other students and it is our professional duty to do everything we can to make it as difficult as possible.
	For further ideas on tackling plagiarism, see Stefani and Carroll (2001) and Atkinson (2002: 1.4). See also the following websites: Culwin and Lancaster (2001), Van Bremmer (1995), the JISC project on electronic plagiarism detection (JISC, 2001). 

13.2  Examinations
The subject of exams can raise passions among students: many hate them, while others actually prefer them to the constant pressure that continuous assessment exerts. In this section we consider the pros and cons of exams, and look at the various stages involved in producing and marking exam papers.

13.2.1  Exams and continuous assessment
We can list a number of concerns about unseen written exams. These include:
•	They do not do much to increase students’ desire to learn: many students avoid modules that involve exams, going instead for options they may be less interested in but which are continuously assessed.
•	They are not a good way of alerting students to what they really need to learn: students often have to guess at the assessment agenda, and their preparation and learning can therefore be unfocused.
•	The amount of feedback that students receive about exams is not optimal: scripts are usually regarded as confidential documents, not to be shown to students.
•	Scripts usually have to be marked in a rush and markers are often tired and bored when they get to, say, their 50th script.
•	Writing, marking, second-marking and moderating exams is a very time-consuming business.
•	They tend to favour candidates who happen to be skilled at doing exams: some of the skills involved have little to do with language ability.
•	They force students into adopting surface rather than deep approaches to learning and into rapidly clearing their minds of previous knowledge when preparing for the next exam.
(Adapted from Race, 2001a: 37–39.)
On the other hand, exams do have some clear advantages compared with a diet of pure coursework. We can perhaps summarize these as follows:
•	They treat all candidates the same and thus avoid the problems of plagiarism. 
•	They force/encourage students to sit down and learn material thoroughly.
•	If students’ work is mostly continuously assessed, there is a tendency for them to concentrate only on assessed pieces to the detriment of their overall learning; this is less likely to happen if assessment is exam-based.
•	When continuously assessed work is returned, there is the strong possibility that students will pay more attention to their marks than to the formative feedback provided by the teacher; with non-assessed coursework there is a greater likelihood of students concentrating on and learning from feedback.
•	Continuous assessment, especially if it involves in-class tests, can seriously eat into teaching time which in some research-led universities is already under considerable pressure; end-of-module exams free up more of the semester for teaching and learning.
On balance, the flexibility of continuous assessment, the opportunity to introduce more varied tasks with potential for positive washback on classroom learning, probably tips the balance in its favour. Furthermore, since some candidates are known to perform below their ability in exam conditions, diversifying assessment allows all students to perform to their maximum potential. However, there is clearly a role for exams both as a summative assessment of language achievement and as a means of assessing students on an equal basis. For this reason most language modules nowadays rightly feature a mixture of the two approaches.

13.2.2  Open-book exams
Essentially these are exams in which students are allowed to take into the exam room reference material such as dictionaries or grammars. Alternatively, or in addition, students might be given FL articles or taped material that they can refer to during the exam. Some open exams may extend over longer periods than the usual two or three hours to allow candidates to work through the material at their own pace.
	Academics tend to be divided on such approaches. Arguments against them include:
•	Exams and in-class tests are not a true examination of linguistic ability if students have recourse to anything other than their FL knowledge and skills.
•	If students are not forced to learn vocabulary and grammatical forms, they will not do so.
•	Some students are likely to bring more or better reference materials with them, thus opening the exam up to charges of not being equitable. And it is unlikely to be possible to buy sufficient copies of standard reference works to supply all candidates.
•	Space limitations mean it is not always possible to find an exam room with sufficiently sized desks for open-book exams.
•	Different standards need to apply if students are to be allowed dictionaries in, say, a translation. This may mean creating additional criteria.
On the other hand:
•	Traditional exams are artificial: in professional contexts, language users will have access to all kinds of paper and electronic reference materials; it is unreasonable to expect any learner to demonstrate encyclopaedic knowledge of the FL. 
•	Open-book exams assess how well students can access and apply information, precisely the transferable information retrieval skills they are likely to need in ‘real-world’ use of languages.
•	They remove the disadvantage that slow writers suffer under in traditional exams.
•	They allow more realistic and pedagogically more valuable integrated-skill activities, such as: listen to the tape and write a response, or write an essay based on the information supplied in the accompanying tape and article.
•	Translation tests with dictionaries shift the emphasis away from the lexical level to meatier discourse-, function- and culture-related translation issues.
The question is thus not a black and white one. As so often in assessment, a mixed economy taking account of purpose and circumstances is probably the best answer.

13.2.3  Setting exams
Writing a good exam paper is an art, dependent on familiarity with the module and its learning outcomes, in particular knowledge of students’ target linguistic level, their knowledge of grammatical structures and the breadth of their vocabulary, as well as on experience of how candidates are likely to understand and interpret questions. You will only get better by doing it, but here are some general notes for guidance:
•	Don’t work in a vacuum: look at colleagues’ previous exam papers for this module. You probably don’t have to follow slavishly what they have done, but this will give you a good idea of the expected standard and syllabus coverage.
•	Fairly obviously, it is important to base exam papers on the type of material covered in the module and not to ask students to engage in any tasks they have no experience of – eg don’t get them to write about nuclear energy in France if the module has only covered the topics of transport, the churches and education, or to translate or do a summary if these skills have not been practised during the course. 
•	As you write an exam paper, keep asking yourself what you are assessing; across a language paper, ensure you are not repeatedly testing the same thing – eg grammar may figure prominently not just in explicit gap-filling exercises but in translations and comprehensions too; the same might apply to a particular body of vocabulary: vary the language and topics featured in the different exercises you employ. Both these questions relate to validity (see 13.1.4.1). 
•	Show your draft exam paper to a colleague so as to get a second opinion on appropriateness of material and level of difficulty, and to enlist a second pair of eyes for proofreading: it is so easy to let typos through, and quite apart from the fact that these look very bad and unprofessional on printed exam papers, they can also mislead students and cause them to waste time puzzling them out. It is obviously better still if your department has a policy whereby teams of staff moderate draft papers and questions.
•	Ensure questions and rubrics are unambiguous, especially where alternative questions are offered, and if the rubric is in the FL make sure in advance that students are familiar with typical terminology (see also Module 14, section 14.4.4 on target-language testing).
•	Run through the questions yourself. For example, if you set a gap-filling grammar exercise and offer a box with possible answers, do the exercise yourself to ensure the number of gaps matches the number of possibilities offered (or, if you don't want an exact match, warn the students on the paper that this is the case).
•	Include a clear indication on the paper of how many marks are allocated to each question (and make sure your sub-sections add up!)
•	For anything other than an objective test (eg multiple-choice), prepare a detailed mark scheme (eg all the points that need to be mentioned in a comprehension task, a summary or an essay) and do this when you write the paper so the material is fresh in your mind, not when you come to mark the papers two or three months later and you cannot recall exactly what you intended when you wrote the exam.
•	If you are preparing a mark scheme that will be used by colleagues, it must be especially clear; you should indicate acceptable alternative answers, show the method for assessing translations or essays by attaching appropriate criteria, and explain any objective scoring method for assessing accuracy (see Module 14, section 14.2.3 on objective marking). 
•	In preparing a mark scheme for essays, it is a good idea to allow more marks than stated in the question as it is very difficult for students to know exactly what you are looking for; so, if you ‘officially’ allow 15 marks for content points in an essay and you specify in your mark scheme the 15 points you are looking for, you might in practice make 20 or 25 marks available for the question so that students who tackle the question slightly differently from the way you have and come up with other points, can still gain good marks. Alternatively, in comprehensions, where there is a finite number of points mentioned in the text but it would be unreasonable (eg because of time pressure in a listening test) to expect students to list them all, you might include in your mark scheme all 16 points but only expect students to list 12 of them.

13.2.4  Moderating exam papers
Most university departments will have some mechanism for moderating the papers students will sit. With finals papers, this will almost certainly involve review by an internal team or committee, followed by comments from the External Examiner. With first- and second-year exams, this process may be restricted to internal review. Regardless of the procedure, the following points are what you should be considering as part of this process (the points inevitably duplicate most of the ones that setters of exams should be asking themselves):
•	Does the exam adequately cover the module learning outcomes, in particular the range of skills involved in the course?
•	Are the texts and tasks of a suitable level of difficulty?
•	Are the timings realistic?
•	Is there any duplication or overlap between topics, skills or tasks across the range of exam components (writing, speaking, reading, etc)?
•	Are all questions clearly laid out and are all instructions appropriately worded?
•	Is the allocation of marks to different questions and components appropriate?
•	Is there a mark scheme?
•	Does it include a wide enough range of possible alternative student responses?

13.2.5  Marking exams
It is clearly crucial to be fair to students when marking exams. But you also need to be able to cope with a large number of scripts, often under intense pressure, so efficiency is another key issue. Race (2001a: 47–49) provides advice on these matters. The following is based on some of his main points:
•	Do not mark too many papers in one sitting and don’t have a daunting pile in front of you on the desk, but stack those you haven’t yet done out of sight.
•	Avoid halo effects: on a multi-question paper, don’t allow a student’s excellent answer to a translation influence you when marking a subsequent essay, with the result that you focus only on the good points and pass over weaknesses. Similarly, after marking a good script, be careful not to be over-critical of the next one.
•	Control your pet hates: don’t allow your dislike of certain things (eg wrong genders, missing accents, absence of paragraphs, repeated misspellings, such as in French *beacoup and *recontrer, or in German  *errinern and *mussen) to cloud your judgement, and make sure these factors do not influence you if there is a ‘benefit of the doubt’ decision to be made.
•	Avoid prejudices: much exam marking is now anonymous but this is not universal. If you do see a student’s name, you must try to be especially objective and banish memories of class and homework performance (both positive and negative ones).   
•	Check back occasionally to scripts you marked earlier to ensure you are applying the same standards; for example, that you are not rejecting a  translation of a word that you earlier accepted, or vice versa.
•	Devise your own system for tackling the marking load: you may prefer to do all answers to Question 1 together or to mark whole scripts at a time. As long as you are vigilant and consistent in the latter approach, it probably makes no difference (although, as noted in 13.1.7, the former approach is likely to save you a bit of time.)
•	As you mark, make a note of how well questions worked and which might have been framed better. This will help when you are setting the exam next time round. At this point it is also a good idea, before you forget them, to put down on paper any implications for the teaching of the module that arise from your marking of the exam.

13.2.6  Double-marking
Consider the following two points of view. Where do you stand on the issue of double-marking?

Tutor voice 1

If you were going to be serious about it, everything ought to be double-marked. That would be the fairest approach….It would also be better for students in that they would get two lots of comments on their work. Of course, time constraints mean it can only be done for exams and substantial pieces of coursework. From the lecturer’s viewpoint, I think there’s no better way to learn how to mark than to do blind double marking with an experienced colleague. That is what I would recommend to all new colleagues: do it with someone who has been at it for years and get them to talk you through why they’ve given the mark they have.

Tutor voice 2

We waste so much time on double marking of language work when the outcome is a difference of a paltry two or three marks. There’s no need to double-mark most language work: usually it’s just right or wrong and I find there’s very little disagreement at all on these exercises, even with more demanding final-year work.

You probably will not have a lot of choice in whether you double-mark or not: it will be laid down in departmental policy. On balance, it is probably a necessary check on standards and consistency, especially for borderline cases in oral and written exams. Just as importantly, it sends a message to students that tutors care about getting it right and being fair. Certain accuracy-based language tasks (eg grammar exercises) probably do not need to be double-marked, and sampling of marking as opposed to blanket double marking is probably adequate for such tasks. But with essays, summaries and free-speaking oral activities there is sufficient room for subjectivity to make double marking strongly advisable.
	Although the second tutor quoted above may feel it is a futile exercise most of the time, there are cases, especially in blind double-marking, where marks fall into different degree categories. This raises the question of what happens after the marking; how does one reach a final mark? Schmidt and O’Dochartaigh (2001: 26) have strong views on this:

Perhaps the worst (though easiest) practice is that of ‘splitting the difference’. Where a small number of marks are involved, and the grade is not affected, then this may be defensible, since both markers have already essentially agreed (eg 55 and 57). But splitting the difference between, say, 54 and 62, thus giving 58, is an abdication of responsibility on the part of the markers. In essence, such a major difference suggests that clear assessment criteria have not been agreed.

Splitting the difference within a degree category is, in practice, the norm, since to do anything else might be taken to mean the opinion of one marker weighs more than the other. While there may be reasons for accepting this in modules where one colleague is more expert (eg a paper on C18th drama), it might be thought contrary to the spirit of double-marking in language work. The problem of differences across degree categories is, however, a far more serious one since in a profiling system, as opposed to a straight averaging of marks, the degree category awarded could affect the student’s final degree classification. In such circumstances, divergent marks deserve to be closely considered and it is the tutors’ duty to examine the piece of work again together, to work through the criteria once more and to discuss any problematic issues. If they still fail to agree, they ought to seek a third opinion.

13.3  Alternative forms of assessment
In this section we consider what are probably less familiar approaches to assessment. First, self-, peer and group assessment, the common factor in which is a desire to involve students more closely in the assessment process. And second, computer aided assessment which is likely to become an increasingly significant feature of language assessment over the coming years.

13.3.1. Self-assessment
The move towards assessment procedures in which students evaluate their own language knowledge, skills and performance is part of a more general move in teaching and learning towards increased learner responsibility and student-centred learning. Its greatest benefit in either sphere of endeavour is its consciousness-raising value. In seeking to evaluate their own work as honestly as they can, students need to step back and take an objective look at their performance. This can help to increase their awareness of both strengths and weaknesses in their language skills and of gaps in their knowledge of language forms. (See also Module 9, section 9.4.1 on ‘Self-evaluation’ in writing tasks.) 
	There are clear benefits for the tutor as well in student self-assessment. If a self-assessment sheet is filled out at the time the task is submitted for marking, this can provide useful information on key issues of concern to the student which written feedback on the exercise might profitably concentrate on. It can also save tutors some time and effort: if students clearly know something already about their performance on the piece of work they have done (eg ‘I had to rush the conclusion, so this is not great’), then there is little point raising this in feedback. Similarly, time can be saved by asking students what they would do differently if they had the opportunity to do it again; often students already know some of the things we are likely to tell them.
	The question on self-assessment forms that most raises passions is the one that asks students to award themselves a mark. Sceptics question whether students can ever judge their own linguistic ability with any real degree of objectivity. As with other disciplines (cf Race, 2001a: 101–02), attempts to investigate this have usually produced reasonable correlations of students’ self-assessment with more objective judgements (eg standardized tests) and with tutor evaluations (eg Dam and Legenhausen, 1999; Bailey, 1998: 227–42). There would therefore seem to be grounds for including self-assessment at least as an accompaniment to more traditional forms of teacher assessment, although your departmental culture and policy will dictate whether it should ever become part of a formal mark for a module.
In setting up any scheme, tutors need to ensure that students have confidence in the process and are not threatened by it. Boud offers sound advice here. He says students need:
•	a rationale: a clear indication of the purpose of the activity;
•	clear procedures: what exactly is required of them?
•	reassurance that anything they reveal will not be used against them in any way;
•	confidence that all students will contribute equally and honestly to the process (Boud, 1995).
Furthermore, one should add that if it is going to work, self-assessment needs to be carefully prepared and students need feedback on how they are doing in assessing their own work (see Brew, 1999).

Activity 2
Below is a sample self-assessment questionnaire about writing. It has been used on a post-'A’ level German language course. 

This questionnaire aims to provide you and your tutor with feedback on your written performance on the German course this year. Tick the box that best describes your feelings about this, using the following scale: 
1 = strongly agree  through to  5 = strongly disagree.

1.	I can now write more freely in German
						
2.	My written German is now more accurate						
3.	I am getting better at organizing my ideas						
4.	My essay structure has improved
						
5.	In my conclusions I am getting better at linking my principal ideas with arguments presented in the main body of the essay						
6.	I am better able to exploit German source materials in my essays						
7.	My use of German idiom has improved						
8.	I am using a wider range of German vocabulary						
9.	I am using a greater variety of structures in my written German						
10.	I am not making as many spelling mistakes as I used to						
11.	I am getting better at identifying language errors when I revise my work						
12.	I have a clear procedure for reviewing my essays once they have been marked						

Are there any aspects of FL writing not covered in it?

Click on ‘Commentary’ for feedback.

Activity 3

Now try to devise a self-assessment questionnaire of your own for speaking skills OR translation, for use with one of your language classes. Show the form to a colleague and see if you have omitted any aspects of the activity. Try it out at the earliest opportunity, maybe the end of a semester. How did your students’ evaluation match your own assessment of their ability? Try to find time to talk through the various issues with your students.

13.3.2  Peer assessment
A form of assessment only slightly less contentious than self-assessment is peer assessment. Many reject it out of hand, claiming students want an authoritative verdict. Indeed, in a series of interviews with students and staff in modern languages, Schmidt and O’Dochartaigh (2001: 34–35) sought views on the role of peer assessment in the subject. Students seemed to have little understanding of the purpose or benefits of peer assessment. Most were reluctant to criticize fellow students in public and some objected to the whole idea, suggesting they wanted the view of an ‘expert’ on their work rather than that of a classmate.
	However, there are clear advantages to be identified. When students get to grips with marking criteria and have to apply them to mark others’ work, they are able to see into our minds a bit better and understand more clearly what we expect of them in terms of performance (Race, 2001a: 94). It is often said that trying to teach something is the best way to get a firm grasp of the topic. The same might be said of assessing someone’s work, since in seeking to judge it, you develop and enrich your understanding of the topic, especially if you are involved in assessing several pieces on the same subject. Equally, especially in such accuracy-based subjects as languages, students can learn a lot from looking closely at work which is better than their own and thinking about why it is superior.
	Peer assessment can work particularly well with class-based oral work. For example, students might be divided into pairs or small groups to perform a task (eg role play, information-gap activity, debate). While Pair or Group 1 and 2 perform the task, Student 3 or Group 3 observes the performance and uses a sheet prepared by the tutor to check if the task is being carried out properly, for example to see whether key information is mentioned or key language is being used, and to comment on their perception of each speaker’s  performance (pronunciation and intonation, independence, fluency).
	Whole classes might also be involved in assessing (anonymously) individuals’ performances in an oral presentation to the group, or else sub-groups might be asked to look at, for example, fellow students’ written work and to assess it according to certain criteria. (I once did this with a post-‘A’ level first year group and their provisional marks were remarkably close to my own. There is in fact research evidence to support this claim for the reliability of peer marking – see Race, 2001b.) The mere process of having to check work against criteria and of comparing it informally with their own efforts, can help students to become clearer about expectations and to identify effective approaches to tasks, such as (in speaking) certain communicative strategies or (in writing) useful arguments, idioms or expressions.
	In spite of this, one might well feel that peer assessment is more problematic for accuracy-based exercises in languages. Indeed in the survey mentioned earlier, staff felt peer assessment was better suited to tasks in which transferable skills such as presentation, negotiation or persuasion were involved. There was also a feeling that students’ marking may need to be moderated in some way in case they are being too strict or too lenient. It was further suggested that if the marks awarded were too far removed from the tutor’s own estimation, they should be altered, which rather begs the question: why bother with it in the first place? As Race (2001b: 13) notes, if tutors are too heavy-handed in moderating self- (or peer) assessment, students tend not to put their hearts into being objective. Language tutors generally felt that for peer assessment to be effective, tutors needed to:
•	ensure clear goals and objectives;
•	work out with students (or provide) assessment criteria;
•	divide responsibilities: students assess presentational and communication skills, staff assess linguistic level and accuracy;
•	average the marks awarded by the group to each piece of work (possibly eliminating the best and worst one or two marks in each case to avoid bias or extreme views);
•	be prepared to invest considerable time and effort in setting up peer-assessment (Schmidt and O’Dochartaigh, 2001: 34–35).
With reference to this last point, it is popularly thought that peer assessment saves tutors time. Experience would suggest this is not always the case. It takes quite a bit of time to prepare assessment sheets to guide students and, with written work, to copy pieces of work for students to look at. It also takes quite a bit of time to prepare the group, to explain to them the reasons for doing it and to reassure them about the process. Finally, for the reasons mentioned above, if the final mark is to count towards assessment of the module, peer marks ought to be supplemented by a teacher mark. In the end, peer assessment can be just as time-consuming as traditional tutor assessment. Nevertheless, it does have clear benefits in terms of student engagement in and improved understanding of the assessment process; so, if only on occasions, it is worth including in your assessment procedures. (See also Module 9, section 9.4.3 on peer evaluation in writing.) 

13.3.3  Group assessment
There is often some confusion about what this term means. As Race states, it can refer to:

[…] the assessment by a tutor of the products of student group work, or to the assessment of the product by students from other groups (inter-peer assessment), or the assessment of the product of group work by students within a group (intra-peer assessment), and can include self-assessment by individuals or by the group as a whole of the product they have generated, and/or their respective contributions towards the product. Therefore it is usual for group assessment to involve at least some elements of peer assessment and self-assessment. (Race, 2001b: 4)

Group work is now fairly common in language assessment programmes and most often culminates in a group presentation of some sort, either to a whole class or recorded on video. In most cases, such tasks are simply marked conventionally by the tutor. A way to heighten motivation by involving a degree of self- and peer assessment is to have students award marks to their own group, including themselves, for different aspects of the task (eg presence and voice projection, use of notes, presentational skills, etc). Alternatively, the whole class might be invited to allocate these marks to each group in turn. Clearly, the tutor would have to moderate marks and supplement them with his or her own assessment of accuracy, fluency, pronunciation and intonation, etc).
	Schmidt and O’Dochartaigh (2001: 93) mention a variation of this approach, whereby group presentations and a related written report are marked by an equally weighted combination of:
•	class members’ mark;
•	group members’ mark;
•	teacher mark for presentation;
•	teacher mark for the submitted report.
	The reason for these involved processes, and the reason why many reject group work, is the issue of the ‘passenger’, or the student who does not pull his or her weight in the group. There is no easy solution to this, but Race suggests possible approaches:
1.	Use the same group mark for all.
2.	Divide the task and assess each component separately.
3.	Award an equal mark to each group member and add marks for individual assessment tasks.
4.	Award a mark for the product and ask group members to peer-assess an additional mark for their individual contributions.
5.	Have a group viva and award individual marks.
6.	Award all group members the same mark but add an individual viva.
7.	Award a single group mark to all and add a separate but related assessment component to a subsequent exam. (Race, 2001b: 17–18)
All these have disadvantages. (1) does not really address the problem and can lead to student resentment.  (2) and (3) may be difficult if the tutor has little insight into the process that has led to the end product. (4) is potentially problematic but does put the onus on those who know best how much each individual has contributed to the task. (5) is fraught with the same difficulties of all group oral work (eg ensuring all get to speak a reasonable amount, that the more able students do not dominate). (6) and (7) are perhaps least contentious but clearly add to a tutor’s workload. 

13.3.4  Computer-aided assessment
This is an area of assessment that is developing all the time as new technologies are refined and adapted to the needs of the language curriculum. The term computer-aided assessment (CAA) can denote a range of activities, summarized by Atkinson (2002) as follows:
•	interactive exercises on a computer;
•	use of Web pages to set tasks and provide support for them;
•	use of computers to complete coursework;
•	on-screen marking of students’ word-processed writing;
•	use of e-mail to send and receive marks and feedback;
•	use of a spreadsheet or database to keep a record of marks;
•	use of plagiarism detection software.
	The justification for the use of CAA is closely linked to the ideas discussed in section 13.1.3 on learning outcomes. If we are using elements of information and communications technology (ICT) in our teaching and one of our objectives is that students should demonstrate command of ICT-specific language skills, then in the interests of alignment of outcomes and assessment procedures, we ought to be using ICT to some extent in the latter. This is a further example of the need for content validity in our assessment (see section 13.1.4.1). Similarly, if computers are being used extensively on the course, eg for grammar practice activities or word processing of essays, and if they fail to feature in course assessment, there can be a negative washback on teaching and learning: students are less likely to pursue opportunities for supplementary computer-based work and tutors are less likely to feel the need to introduce ICT into the classroom (see section 13.1.4.5 on washback). 
	Owing to the logistical demands of employing CAA in summative assessment, which include the crucial ones of practicality and security, the main application of CAA for the foreseeable future is likely to be in formative assessment (see section 13.1.5). A major advantage of CAA for formative assessment is that feedback can be given immediately and, with the more sophisticated software, is accompanied by explanations of why certain choices are inappropriate. However, it is true that many CALL (computer assisted language learning) applications still fall well short of this type of ‘intelligent’ feedback as it is so time-consuming, and therefore expensive, to produce. (See section 13.5.2.1 on other uses of ICT for feedback.) 
	Atkinson (2002) summarizes typical exercises types which include:
•	matching / pelmanism;
•	multiple-choice;
•	Cloze (see 14.5.1); 
•	sentence reordering / jumbled words;
•	text reconstruction;
•	true or false;
•	crosswords;
•	wordsearch;
•	text entry quiz.
CAA is most likely to be helpful in assessing listening, reading, and knowledge of grammar. In typical listening and reading packages, multiple-choice comprehension questions will be accompanied by feedback on each of the responses, thus providing not just the correct answer but also valuable incidental learning opportunites. Longer reading texts are less suited to CAA owing to the difficulties for the student of processing large amounts of text on screen, and, similarly, discursive questions in both listening and reading that require more open-ended answers are not possible. Grammar is the area that is most highly developed in CAA with any number of exercises available on the Web and on commercial CDs in a wide range of languages. Some are frankly disappointing and, from the student perspective, no more motivating than traditional pen-and-pencil grammar tasks. Increasing numbers, however, feature the best aspects of feedback mentioned above and, when linked to coursework, can serve as a genuine motivator of students for whom formal grammar is a challenge.
An area of CAA in which huge advances have been made in recent years is that of diagnostic testing. The principal focus recently has been DIALANG, an EU-funded project that seeks to diagnose language competence in 14 EU languages, using online assessment in listening, reading, writing, structures and vocabulary, and making extensive use of both placement and self-assessment tests (DIALANG, 2002). A Pilot Tool and Pedagogical Pack can be downloaded from (http://www.dialang.org/english/pedapack). By way of feedback, the materials also provide advice on how learners can improve their competence  to the next level in the Common European Framework levels of language proficiency (Council of Europe, 2001), ranging from A1 (Breakthrough), through A2, B1, B2, C1, to C2 (Mastery).

13.4   Understanding language errors 
In this section we consider the nature of error in language learning, including the difference between slips or lapses, errors and mistakes. We also explore why students make the mistakes they do, and how tutors can best respond to errors. 

13.4.1 Cause for despair?

Tutor voice 3

[Lecturer of French in a pre-1992 university]

Maybe I’m just getting old and cynical, but the quality of language produced by students these days is incredibly poor – especially first- and second-years, but also some finalists. I really get fed up with constantly correcting the same basic error time after time…I sometimes wonder what secondary schools are teaching kids…I don’t just mean most of the group make the same errors but also that an individual student will keep making the same stupid mistake and no matter how often I correct it, he or she will just keep on doing it, week after week.

Leaving aside the debate about falling grammatical standards (see Sheppard, 1993; McCulloch, 1995; Klapper, 1997, 1998; Gray, 1999), this comment about students’ apparent inability to learn from their errors echoes many a moan over coffee in university common rooms. Schmidt and O’Dochartaigh (2001: 31) quote the strikingly similar view of one of their interviewees, a lecturer in German: ‘It really frustrates me and makes me wonder why I’m here, when a student, no matter how many times I point out that it’s wrong, gives me an English plural in German such as ‘Die Politikers’. Sometimes I wonder who let them onto the course!’  So, does the linguistic fallibility of today’s students signal the end of civilization as we know it? What are we to make of this apparent incorrigibility? The answer is to be sought in the study of second language acquisition.

13.4.2  Errors and second language acquisition
All language tutors will be aware that learners frequently produce incorrect forms when they in fact know the correct ones. It is the tutor’s task to distinguish ‘errors’, the result of deficient knowledge, from such ‘mistakes’ which are merely occasional slips or lapses (Corder, 1967). For a detailed discussion of the nature of error and a review of the latest writing on the topic, see James (1998). Here you will also find the following classification of ‘linguistic deviance’, which represents a helpful refinement of Corder’s distinction:
•	Slips/lapses: these can be readily found and corrected by students themselves (eg when checking through a piece of work).
•	First-order mistakes: these can be corrected by students themselves if they are prompted, or alerted to some shortcoming in their output (eg a mispronounced word is repeated back to them by the teacher).
•	Second-order mistakes: these can be corrected by students themselves if they are told where they have gone wrong and what the nature of the problem is (eg if a word is underlined and ‘subjunctive needed’ is written in the margin).
•	Errors: these cannot be corrected by students until they have been given further information and further learning has taken place.
(Based on James, 1998: 83)
 	Many of the errors that learners make do not resemble either FL or L1 forms. They are also remarkably regular and systematic, and it has therefore been suggested that they must be part of an internal learner system. This ‘new’ kind of language used by learners is often referred to as ‘interlanguage’ (Selinker, 1972). Interlanguage denotes an autonomous language system which is dependent on its own rules. The system changes constantly as learners progress in the language: for example, learners apply a new rule in one context and then gradually learn to apply it in another. 
	A significant feature of interlanguage is its variability (Tarone, 1988). At any stage of their development, learners sometimes use one form and sometimes another; this may involve alternation of one correct and one incorrect form, or even of two incorrect forms – probably the greatest source of tutor despair! Yet the choice of these forms is not entirely arbitrary: after some initial free variation, which may be a necessary stage in acquiring grammatical structures, learners start to use variable forms systematically, gradually improving these forms until they are correct. This systematic variability applies to the development of particular grammatical structures; for example, learners may have mastered the French imperfect tense but their use of ‘de’ and ‘à’ following adjectives may still vary freely. 
	Research suggests that errors are an inevitable and natural part of the language acquisition process. Making mistakes and errors, and learning from them, is an essential element of language learning, which is in essence a process of learners forming hypotheses about target language forms, refining these in the light of evidence from target language usage and employing their reformulated hypotheses to seek further input and extend their knowledge of the FL.
Furthermore, structures are not acquired in a linear, uniform or ‘building-block’ approach, despite what many textbooks and the layout of pedagogical grammars seem to suggest. Certain key structures are subject to an acquisition order and although this process can be hastened through formal work on the language and through targeted feedback, it will always be a gradual, incremental process, with fits and starts, forward spurts and intermittent regression. When you have corrected an aberrant auxiliary verb for the tenth time in a particular student’s essay, maybe you can console yourself with this thought and tell yourself your breath and ink are not really wasted: he or she will get there eventually!
However, it is also well known that many learners never progress to target-language use in certain grammatical structures (Ellis 1997: 29). You can  detect the beginnings of this ‘fossilization’ in advanced learners, in particular. For example, final-year undergraduates still produce basic errors in sentence structure or verb endings – despite many years of tuition; and some older learners, who have been attending advanced classes for years, still make basic errors and have poor pronunciation. In such cases, you need to give serious consideration to how to maintain students’ motivation. If, for instance, after a  lot of practice and correction a student’s pronunciation does not seem to improve, you may want to concentrate on other aspects, such as stress and intonation, rather than try to improve pronunciation.

13.4.3  Why do students make mistakes?
Regardless of whether linguistic shortcomings are classed as mistakes or errors in accordance with Corder and James, the teacher also needs to consider the underlying cause of them. This is likely to be one of the following:
•	inadequate knowledge of a rule (eg not knowing that Russian requires the genitive for animate direct objects);
•	a rule wrongly applied (eg getting the order of French object pronouns wrong);
•	over-generalization of a rule (eg putting the verb to the end of all German subordinate clauses);
•	interference from L1 (especially useful when students share the same mother tongue); this can be one of the most productive ways to focus FL learners’ attention on form (see Module 3). 
Knowing the cause of deviance will help you to decide how to deal with it in feedback.

13.4.4  Responding to errors
To be confronted repeatedly with the sort of basic errors mentioned by the tutor at the start of this section is clearly a demotivating experience for the tutor. But it should not be forgotten that it is equally demoralizing for the student who has failed yet again to come to terms with the problem and master the correct form or rule. These situations are a challenge to our pedagogical ability. They require a review of how we provide feedback, how we explain the errors students have made. Questions to ask oneself include:
•	Are my explanations clear enough?
•	Am I raising too many issues/grammar points simultaneously?
•	Am I confusing students by not isolating the point of grammar or whatever?
•	Am I using metalanguage which students are not sure about?
•	Am I using illustrations that are close enough to the original problem? Do they include needlessly difficult (and therefore distracting) vocabulary?
•	Can I try to get the student to reflect on why he or she is making the errors? (This may provide a ‘handle’ or possible approach.)
As the last point suggests, such situations challenge tutors to ‘get down to the student’s level’, to see the problem through the eyes of the student. With weaker students, in particular, this is very difficult for able linguists to do; it is likely to involve some form of one-to-one discussion with a student and, if your university has one, reference to a Language Advisor who can work through with the student the problems he or she is having.

13.5  Feedback and error correction
In this section we look at ways in which you can provide students with comments on their work. We then consider what lessons can be learnt from different approaches to correcting written and oral work, and discuss the key role of self-correction. (See also Module 9, section 9.4 on ‘Suggestions for correcting and giving feedback on writing‘.) 

13.5.1  Providing feedback
Before you start this section, read the following comments taken from samples of marked work. How effective do you feel they are? How much information or help do they give students? 

“Your grammar is very weak.”

“You have a very narrow range of vocabulary.”

“This is simply not good enough.”

“Quite a good effort.” 

“On the whole, a very good piece of work.”

There is a danger with feedback that it will be perceived by students as final and damning, suggesting that there is no hope of sufficient improvement to redeem the situation. (Besides being far too brief, the first three examples above fall into this category). Feedback, even on the weakest piece of work, should hold out some prospect of the student being able to retrieve matters, otherwise why should he or she bother? This is what Race (2001a: 87) calls feedback ‘oriented to opening doors, not closing them’. Similarly, able students will not be able to progress further unless we tell them what is good about their work and which aspects can be improved still further. (The last two comments in the box above are fairly meaningless for this reason.) 
	In brief, feedback needs to be:
•	specific, ie give particular examples of what was wrong and what was good;
•	clear, ie use unambiguous comments, however brief, rather than inexplicable ticks or squiggles; if you use a system of annotations (eg Appendix 4 – and see also Module 14, section 14.2.5), be sure students have a copy of and understand the scheme;
•	remedial, ie indicate how things can be improved; but obviously this needs to be realistic as it is unusual for a fail mark to become a first next time round;
•	prompt, ie as close as possible to the date of submission so students can still recall the process of doing the task and their thoughts or intentions during it;
•	flexible, ie students may want to talk to you in person about certain things and there needs to be some mechanism for this;
•	personalized, ie most of your comments should be specific to the individual student’s work; even if you hand out a model version, each script ought to receive personalized comments.

Now consider the following two extracts from an interview with a school teacher and a cri de coeur from a student. What issues do they raise for us and how would you respond to them? Click on the ‘Commentary’ button  following the extracts for some feedback.

Tutor voice 4

[A secondary teacher of modern languages, a former graduate of French and German from a red-brick university]

The thing I always remember about assessment at university was how we, or at least I, always seemed to get the same sort of mark, no matter what….My translations were almost always in the range B+, or 64–68, and it seemed no matter what I did, no matter how much effort I put into an exercise, I always came out with a bloody mid-2i! Maybe I was just very consistent….yet there were times when I thought I’d done really well but I never seemed to be able to slip into the never-never land of the first class…..What’s more, we were never told how we could get up a grade. If you asked, it was basically: ‘Be more accurate’, or ‘Pay more attention to detail’, which was pretty useless...that was what I found really frustrating….

Tutor voice 5

When I’m marking, I sometimes find myself thinking back to the way our lecturers used to mark our language work…. My kids get corrections written in, a brief comment at the end and a letter relating to a standard grid that tells them what they have to do to improve next time…. At university, we didn’t even get this basic feedback…at most, it was a few scribbled corrections between the lines and a one- or two-word comment at the end – and that in a class of 8–12 students! What’s more, my lecturers were teaching no more than 7 or 8 hours a week, while I have 22 class contact hours each week and no class is smaller than 25!

Commentary 

Student voice 5

[Final-year student of Spanish]

I find lecturers don’t have as much time to give us feedback on our work as I’d like….Maybe there’s nothing can be done about it….I know they’re busy…but I’d like to be able to ask some of my lecturers about assessment, about how they’ve marked the stuff, you know…especially assessed coursework…and not feel they think I’m, like, having a go at them or disputing their marking, or challenging them in some way. Another thing is that.… you know…. some lecturers just point out the mistakes…. and all you get back is a mass of red pen with no recognition that you’ve at least got some of it right….

Commentary 

When giving feedback, it is important to avoid both excessive metalanguage (some students may not understand it fully, even though you may have explained it to them) and sarcasm (not everyone will share your sense of humour). It is also a good idea to try to balance negative comments with something positive on at least one aspect of the work. Indeed, comments might usefully start with the positive. Consider the following, for example, taken from an essay written in a Department of German and awarded a low mark of 45%:

You address the title effectively and have some interesting things to say about the roles of the President and the Chancellor. You also try to use quite a wide range of vocabulary. However, there are serious shortcomings in your use of grammar. In particular, you need to look at all the errors you have made with the passive (marked ‘P’) and word order (‘WO’). You are also inconsistent with case endings; I have indicated these with ‘K’ but have left you to correct them yourself – ask if you are not sure. Your sentence structure is very repetitive (too many instances of ‘es gibt’ and ‘auch’). To get this into the 2ii category you need to iron out the problems with case endings and word order, and introduce more variety into your sentence structure – see my handout on ‘Idioms and Structures for Essays’.

It is a moot point whether feedback such as this should always, sometimes or never be given in the target language. Departmental policy and individual preferences will vary on this. If it is done consistently and typical terms are explained to students early on in the course, there should be no problem. Clearly, the level of the group’s proficiency will dictate how much students will understand at first but the more it is done, the more readily students will adapt to it and accept it as part of normal target language use. (see also Module 14, section 14.4.4, on target language testing.) 
One time-honoured approach to providing feedback is to link written correction and annotation of individual pieces of work with whole-group oral feedback, in which recurrent or particularly problematic aspects of performance are dealt with collectively. Discussing common weaknesses in this way is an effective method of providing formative feedback, but it also avoids personalizing negative comments and thus embarrassing individuals. Students will recognize which errors are theirs and take note of comments, but they can do so in a safe, anonymous environment. 
An extension of this approach is to provide an unattributed list of common mistakes and corrections on a sheet for distribution to the whole group. Alternatively, corrections could be omitted and the group required to correct all the errors themselves, possibly in pairs. This can be an effective learning experience, although it is important to ensure clear and comprehensive correction is provided to all students so they are not left in possession of incorrect exemplars of the target language. 
Finally, Race (2001a: 88–89) provides a lot of good advice on the practicalities of feedback and you are advised to read this in full. The following summarizes some key points:
•	Make your writing legible. If there is not enough room for comments on a student’s work, use a code system and write feedback on a separate sheet (see Appendix 4 for an example; see also Module 14, section 14.2.5, for further examples). 
•	When you collect their work, give students an initial feedback handout (eg a model answer) while the task is fresh in their minds; then concentrate your marking on individualized feedback.
•	Be honest: it’s no good withholding the truth of their weakness from students. But if there are serious problems, eg a student is heading for failure on the module or is guilty of plagiarism, it is better to deliver such news personally so you can monitor how it is being received and offer support.
•	If you feel students are not taking sufficient note of your feedback comments, return the work minus a mark and get them to work out an appropriate mark by the time of the next class, thus inducing them to read and reflect properly on all the feedback.
•	Consider using audio tapes for feedback. In languages this might be especially convenient when marking oral tasks submitted by students on tape. Students like such personalized oral feedback and, since you can talk more quickly than you can write, you can probably give better quality feedback in this way, as well as modelling correct linguistic forms for students to listen to.

13.5.2  Correcting errors in written work
This is a topic that few tutors pay much attention to, often simply adopting techniques they have experienced themselves as students. Yet, if we believe that errors are central to the way students learn a language, it is a crucial area that deserves our attention. Before looking at different approaches, it will be helpful to clarify your current approach.

Activity 4
Which of the following fits most closely your approach to correcting errors in students’ writing?

1.	The tutor underlines and corrects only those errors which are the most serious or which impede communication of the message.

2.	The tutor underlines errors and writes in correct versions above the line.

3.	The tutor indicates that an utterance is inaccurate by underlining the error and then corrects it in oral feedback to the class.  

4.	The tutor indicates that an utterance is inaccurate by underlining the error but leaves the student to correct it.

5.	The tutor indicates that an utterance is inaccurate by underlining the error, gives a further prompt, eg by writing in the margin what sort of error it is (word order, tense, agreement, etc), and then corrects it in oral feedback to the class. 

6.	The tutor indicates that an utterance is inaccurate by underlining the error, gives a further prompt, eg by writing in the margin what sort of error it is (word order, tense, agreement, etc), but leaves the student to correct it.

Click on ‘Commentary’ for feedback on these options.

Apart from these traditional approaches to error correction, you might consider a number of classroom activities such as the following, which aim to develop the ability to self-correct by encouraging analysis of errors and reflection on their cause.
•	Give students back their corrected work and have them analyze their errors to decide which of the following three categories they belong to:
(1) mere ‘slips’ of the pen or silly mistakes;
(2) ‘mistakes’ they could have corrected themselves if they had been 
prompted;
(3) something they were sure was right (ie an ‘error’).
•	Get students to work in pairs or groups on uncorrected pieces of FL writing  (not their own); their task is to identify errors and correct them. The exercise might be varied or made a little easier by telling them there are, for example, five errors in paragraph 1, seven in paragraph 2, etc.
•	Give students an essay with the errors underlined and annotated, as in points 5 and 6 in Activity 4; they have to find the correct forms.
•	Hand out an unmarked piece of work and ask students to identify a specific number of errors – eg six problems with word order, ten with  agreements, three wrong tenses, etc. 
•	Give the class a piece of unmarked but incorrect FL writing, along with a corrected version. By comparing the two versions, students have to identify the errors and decide what sorts of error they are. Subsequently, they have to speculate on why the error might have been made.
(See also Module 9, section 9.4.2 on ‘Exploring alternatives to correction’.) 

13.5.2.1  Feedback and ICT
With students nowadays submitting much of their work in typed and electronic formats, a useful new way to provide feedback is to employ the marking tools  available in word processing packages. This means students can submit work electronically and the tutor can mark it without having to first print it out and can also return it electronically. See Figure 13.1 for an illustration of this based on an extract from a short essay written by an intermediate student of German:

Mein Sommerurlaub 

Mein Freund und ich waren diesen Sommer in Koln Köln und wir hatten ein unangenehmes Erlebnis, denn wir waren von Taschendiebe Taschendieben am Bahnhof beraubt.
	Wir waren gerade nach langer müder ermüdender Fahrt aus dem Zug gestiegen  ausgestiegen und mein Bruder, der die Koffer tragte  trug, folgte mir. Aber weil wir den Bahnhof nicht kennten kannten, stellten wir die Koffer unten ab, um uns in an der Information nach einem Hotel zu erkunden erkundigen. Mein Bruder setzte setzte sich auf eine Bank und las die Zeitung, während ich zum Informationsbüro ging……..

Figure 13.1  Example of on-screen marking and feedback

The words and phrases highlighted in yellow above contain comments that appear as the mouse is passed over them. (An annotated version of this diagram can be found at the end of the module.)
This on-screen editing is very simple to set up in Microsoft Word (1997): 
•	Bring up the Reviewing toolbar (View/Toolbars/Reviewing)
•	Click on Track Changes on the Reviewing toolbar
•	Type corrections directly onto the text or, to give for more detailed feedback, use the Insert Comment icon on the Reviewing toolbar.
Additional features are available in the Word 2000 version. 
In providing detailed feedback the tutor can choose either to give the correct form and an explanation, as in the second and third highlighted examples in Figure 13.1, or just an explanation, as in the first example, leaving the student to self-correct and maybe submit a second draft at a later date.

Activity 5
Now have a go yourself. Use the on-screen editing tool in Word to mark and provide feedback on the following letter of application for a job as a holiday rep., written by a student of English as a foreign language.

Dear Sir/Madam,

I write about your advertisement in Guardian newspaper to find student rep for the summer works.

I am all those things you write about and in particular am get on well with all people I know. I work in supermarket on evenings after my study. I French and Spanish speak very good and little Portuguese. I have last summer worked hardly in tourist office in Barcelona and so having had expereince of such work.

I apply for this job beause I am loving your country a lot since I go on holiday there with my family and because I want in the future to do work in tourist industry and this will give me abundant expereince. Also I like work with young people in all ages and size and I think I will make up a good group leader for young groups.

I send with this letter my CV. I hope you like my application and I look forward to hearing your reply.

Yours faithfully,

The major objection to this approach to feedback is that it can be quite time-consuming to type in different text for each individual student. One way to reduce the burden is to use the type of  agreed system of annotations (V for verb, P for passive, etc) mentioned earlier; see, for example,  Appendix 4, and also Module 14, section 14.2.5. 

13.5.3 Correcting spoken errors 
Although correction of spoken errors is not central to the concerns of this module, there will be cases where you are inclined to correct student oral production in an assessment (eg an oral), especially if it will help the student to express himself/herself with greater clarity. More frequently, it is a crucial skill in language teaching and it is therefore worth a brief review.
	What you need to ask yourself before jumping in and correcting a student is the following:
•	Should I simply ignore this (relatively minor) error?
•	If I intervene at this point to correct, will it interrupt the student’s ‘flow’ and inhibit him/her?
•	Is this an error he/she can quickly self-correct when it is pointed out?
•	Will it be bad for the student if I do not correct the error?
•	Will it be bad for the rest of the class to hear uncorrected forms?
The answers to these questions will depend on the situation, the nature of exchange, the students themselves and other factors such as time constraints. There is thus no easy answer to any of them.

Activity 6

Think back to your own language learning. Did you enjoy being corrected when speaking the FL? Try to recall how your teacher(s) corrected your oral production.

Compare your own approach as a tutor with that of your former teacher(s). Is it similar to theirs?

Another major factor is the tutor and his/her conception of what a good teacher does. As Module 1 suggests, we are very often shaped by our own experiences as learners and how we correct seems to be one area which is especially influenced by our teaching ‘apprenticeship’, ie the time we spent in the classroom as learners. 
	Research has indeed shown (Kleppin and Königs, 1991: 273) that tutors have certain set preferences when it comes to correcting students and that they employ these independently of the type of activity (eg grammar exercise or free oral production). To help you become more aware of these preferences, you may like to try Activity 7.

Activity 7
Fill out the following questionnaire about correcting students’ free oral work. Be honest: make sure you say what you actually do in class, rather than what you think is the ideal approach:

1.	When students are speaking freely in the FL, do you correct a lot or not at all? Ring the appropriate number on the scale:
	A lot 	1	2	3	4	5	not at all

2.	What do you correct mainly?
	  pronunciation
	  grammar
	  vocabulary
	  content

3.	Who does the correcting in your class?
	  the tutor
	  the learner who made the mistake
	  the other students

4.	When do you usually correct?
	  immediately after the error
	  at the end of the completed utterance
	  in a subsequent correction phase of the lesson

5.	Do you encourage self-correction?
	Yes, very often   1	 2	3	4	5	no, never

6.	If you provide direct corrections yourself, how do you usually correct?
	  I give only the correct form so it is clear what the mistake was
		  I repeat the whole utterance, with corrections, in an incidental 
	     way

7.	Which language do you use when correcting?
  FL only
  L1 as well as FL
  I try to bring into my explanations other languages that
	     students have learnt

8.	What tone do you use when correcting?
  	  friendly     reproachful     ironic     humorous     neutral

(Based on Kleppin, 1998: 75–77)

Now click on ‘Commentary’ for feedback. 

13.5.4  Self-correction
As the previous two sections have suggested, correction must not be the sole preserve of the tutor. With the increasing attention being paid these days to learner autonomy (see Module 12), we need to be doing all we can to encourage students to self-correct for James’s slips/lapses, first- and second-order mistakes. 
However, we should not just see self-correction as something that happens after the tutor has marked a student’s work. Drawing on Graham’s (1997) taxonomy of self-correction and prompted correction processes, Powell lists a number of strategies that students can be encouraged to adopt before they submit language work, including:
•	checking a completed task for meaning: has the task been properly addressed and fully executed?
•	checking for grammatical accuracy;
•	re-checking a piece of work after a certain period of time to gain some distance from it, possibly using ‘read aloud’ techniques;
•	reviewing strategies and planning involved in the task and how they might be improved.
(See Powell, 2001a: 148–49, for a fuller discussion.)

13.6  Summary
As a summary of the issues discussed in this module, you will find below a checklist in the form of questions which we need to bear in mind when planning assessment procedures for language modules:

1.	Does the module feature a mix of formative and summative assessment?

2.	Is there an appropriate blend of continuous assessment and class tests/exams?

3.	Are the assessment criteria clearly linked to learning outcomes?

4.	Are students involved sufficiently in the assessment process?

5.	In particular, have students been given a copy of the scheme of assessment and the marking criteria?

6.	Where appropriate, has consideration been given to opportunities for self-assessment?

7.	Is there any scope for involving students in peer and/or group assessment?

8.	How do individual assessment tasks measure up against the demands of validity, reliability, transparency, practicality and washback on teaching?

9.	Do the assessments cover a sufficiently wide range of topics, contexts and language?

10.	 Why are individual assessments being done and what precisely is being   tested?

11.	 When will assessments take place?

12.	 How are students to be assessed (individually, in pairs, as a group)?

13.	 Are titles of essays sufficiently specific to reduce the likelihood of plagiarism?

14.	 Are tests or exams to be conventional or of the open-book variety?

15.	 If open-book, which materials are students able to bring with them?

16.	 Does the department need to provide any reference material for the exam?

17.	 Have exams papers been appropriately moderated and have arrangements been made for double-marking, where required?

18.	 Is there an unambiguous mark scheme, preferably one produced at the same time as the exam or test?

19.	 When providing feedback on students’ errors, is the tutor bearing in mind the likely cause of errors?

20.	 Does marking take account of the difference between ‘slips’, ‘first- and second- order mistakes’ and ‘errors’?  

21.	 Is there an agreed system for short-hand annotation of student work and do students fully understand it? 

22.	 Is there a consistent policy on how much correction of written work is to be done by the tutor? And do students understand this?

23.	 Is there some system for following up or reinforcing student corrections?

24.	 Are marking criteria clear and activity-specific?

25.	 Is there an appropriate system for providing a detailed response to written  work (eg a standard feedback or comments sheet)?

Activity 8

Now go back to Activity 1 and reformulate your 10 beliefs on assessment in language teaching in the light of your work on the module. Have you changed your mind about any of the 10 items? If possible, compare your responses with those of a colleague who has also worked on the module, and discuss any issues you are still unsure about. 

13.7  Commentary on activities

Student voice 1
This chimes with the findings of Schmidt and O’Dochartaigh (2001: 21) who state:

Students indicated in interview that if they were given too many assignments, they tended to just ‘get them done’ and move on to the next one. There was a clear suggestion that over-assessing can lead to a surface approach to learning. The students use dictionaries to find the right phrases for the assignment, but they have little or no time to spend actually absorbing those phrases into their active vocabulary.

Quite apart from the implications of this for students’ learning of the language, such over-assessment also inevitably means that teachers have less time to provide students with feedback on the work. This can often only be tackled at departmental level, although if you have responsibility for a module, you can at least start there and encourage discussion of the topic more widely. There can be no doubt, language courses do feature a lot of assessment compared with other modules, and there are good reasons to do with language acquisition why this should be so, but there are ways in which we can reduce the burden for staff and students and, at the same time, make the learning experience of assessment more effective. Computer-aided assessment with built-in feedback has a clear role to play here (see 13.3.4). 

Student voice 2
One response is to provide a module outline at the start of the course, with a timetable of session topics and assessed work deadlines. It may be that your department or school would agree to an extension in such circumstances, although there are those who might think students just need to organize their time better and plan their workload, as they would have to do in most jobs.

Activity 2
Other issues that the self-assessment form might have included are:
•	reference to interesting content;
•	being mindful of target audience (ie who the piece of writing is notionally aimed at);
•	use of paragraphs and essay structure;
•	ability to back up arguments with examples;
•	use of connectors to link paragraphs;
•	in academic essays, use of academic apparatus (quotations, references, etc).

Tutor voices 4 and 5
The pressures on university lecturers have grown enormously in recent years and this graduate’s recollections of, perhaps, cosier times are not applicable to contemporary university life. Nevertheless, we must not ignore the major role assessment plays in effective learning, and the professional duty we have to ensure our students receive constructive and practical feedback on their work. While our group sizes may now be nearer 20 than 12, we really owe students something more than comments such as ‘Quite a good effort’ or ‘Room for improvement’. If appropriate (eg on translations), a ‘correct’ version might be supplied and, to save time having to write out lengthy comments on each piece of work, a feedback sheet such as that provided in Appendix 1 might be employed, including a specific line such as: ‘To move into a higher category next time/to gain an extra 10%, you need to………’ (see also Student Voice 5, section 13.5.1). Alternatively, the type of criteria grids in Appendix 2 might be used with appropriate additional, personalized comments. 
Of course, what students are most interested in is the mark you give them, as the teacher’s recollection of his student days confirms. However, in much of the work students do, the mark is, or should be, of secondary importance. Marking and feedback are primarily about improving the quality of learning, not providing a summative judgement on performance (see section 13.1.5). It may well be that even if we employ a formative approach, students will persist in noting only the mark. The reason for this might lie in issues beyond lecturers’ control (eg the effects of modularization) but this is no reason for not trying to underscore the role of formative feedback by, for example, handing back non-assessed homework without a mark, getting students to focus on comments and corrections, and only revealing the mark awarded after this discussion has taken place.

Student voice 5
Finding some time outside of class when we are available to talk to students is essential. It is advisable to set up regular consultation hours, a fixed time each week when students can drop in to see you without an appointment. Also you might consider an e-mail enquiry day or afternoon a couple of times per semester when students can ask you questions specifically about assessment and marking. 
Some people believe that by avoiding the use of red pen when marking, we show more respect to students’ work and avoid possibly damaging psychological effects. Regardless of your views on the use of red pen versus pencil or less stark, ‘authoritarian’ ink, it clearly is demotivating for weaker students to see purely negative marking, eg a mass of corrections and exclusively critical comments in the margin. It is therefore a good idea to include some comments on positive features – even if these may be rather tokenist in certain cases – and some ticks amidst the underlinings and written corrections. If a student feels he or she has got at least something right, the task of tackling all the problem areas will seem a little less daunting.

Activity 4
1.	Since, as we have seen, errors play an important role in language acquisition, we really should correct or at least mark up all errors in students’ written work. If we don’t tell them, how are they to know it’s not right? And how are they to know what counts as a serious error? Tutors sometimes do not mark all errors out of fear of demoralizing weaker learners. Rather than ignoring errors, it is probably better in such cases to mark positively too, ie to flag up things the student has got right. If a student is very weak and you find yourself having to correct large amounts of text, an alternative might be to focus on one main item (eg verb agreement, gender, adjective endings) and mark this severely and with abundant comment, but to mark other errors in another, less prominent colour or via underlining only.  
2.	As a blanket approach, this is probably the least effective: it increases the amount of red pen on the students’ work, is more time-consuming for the tutor and does nothing to encourage a student to think about the nature of the error and how it might be improved. However, it is a necessary approach in cases where the tutor suspects that even with extra help the student will not be able to self-correct. It is also required when students use inappropriate idioms or expressions, since most are unlikely to be able to correct these themselves.
3.–4. Just underlining errors and encouraging students to self-correct is a good idea if you are confident all students are capable of this in all cases. Usually, however, this will prove too demanding for all but the most able. A useful modification that can help point students in the right direction is to colour-code your underlinings according to type of error. I once had a colleague who employed the following scheme:
•	red for morphological and word order errors;
•	green for vocabulary errors;
•	yellow for punctuation and spelling problems.
5.–6.  Giving students clues in this way can help them to correct many errors themselves (cf James’s distinction of errors from mistakes and slips – see section 13.4.2), while those that they are unable to fathom for themselves can be covered by the tutor in group feedback. This is an especially useful approach with grammar errors. The important thing is that the signs and symbols the tutor uses in the margin are (a) clear and unambiguous, and (b) familiar to the students (see Module 14, section 14.2.5 for sample schemes in French, German and Spanish that can be readily adapted to other languages; Appendix 4 is one that I use in German). A variation on letter annotations in the margin, is to provide a reference to the relevant section of the recommended course grammar, eg ‘see French Grammar, paragraph 82.4’.

Activity 7  
Once again, there is no correct answer to these questions. All teachers have their own way of providing oral correction. There is very little evidence to suggest one particular approach is more effective than any other since there are so many variables involved, including teacher personality and, just as importantly, learner types: some learners will respond better to certain types of correction than others. This questionnaire and what follows are offered merely in the spirit of awareness-raising, providing you with the opportunity to reflect on how you currently correct students and to consider whether there is anything you might like to add to your repertoire of techniques.
	With regard to self-correction, here are several possible approaches (cf Kleppin, 1998: 76–77):
1.	Simply tell the student something is wrong and leave them to find and correct the error (can be very difficult for the student).
2.	Indicate where the error is – eg ‘He has [what] the channel??’ [swum].
3.	Give additional help through metalanguage (eg ‘it’s feminine’, ‘use the accusative’, ‘passive!’)
4.	Let the student know something is wrong through non-verbal means (eg gesturing with hands; frowning; raising an index finger and shaking your head; clicking your fingers; repeating part of the student’s utterance, pausing before the error and raising your intonation – eg ‘As usual, the warnings have been _______?’).
5.	Give the start of a word (‘inter____?’).
6.	Give the opposite of a word (‘not easy, but…..?’).
7.	Remind students when and where they learnt the particular item.
There is also the question of what to do after students have corrected themselves. The options here are:
•	Not do anything else and move on.
•	Get either the student who made the mistake or someone else in the group to repeat the correction.
•	Repeat the correction yourself.
•	Ask for or provide further explanation about the corrected form or expression.
If you correct the error yourself, on the other hand, the options include:
•	Leave it at that and move on.
•	Get a sign (eg a nod of the head) that the student who made the mistake understands, and move on.
•	Get the student who made the mistake to repeat the correction.
•	Or, as above, initiate further explanation.

13.8  Appendices

Appendix 1

Sample Feedback Sheet (Written Work)

University of Birmingham, Centre for Modern Languages

	I        2i        2ii         III        F	

Grammatical Accuracy
Accurate	             	             	Inaccurate

Range of Vocabulary
                             Wide	             	       	Very restricted	

Structure and Content
 Good introduction	             	             	Poor introduction

Appropriate length	             	               	Appreciably under/over 
length 

Sound conclusion	             	             	No/weak conclusion 

Content and Analysis
Factually accurate	             	              	Substantially inaccurate

In-depth discussion	             	              	Superficial discussion

         Very good analysis	             	        	Poor or no analysis					

Spelling and Punctuation
Accurate	             	              	Very inaccurate

Use of Sources
Evidence of research/reading	             	               	Very little evidence

Appropriate use of quotations/	             	               	Poor/inappropriate 
citations 							use of quotations/ 
citations
Comments:

Overall grade awarded:

To move into a higher category next time you need to:

Appendix 2

A.  Criteria for Assessing Long Essay

University of Birmingham, Department of German Studies

Name:_________________________________

	A (100-70%)	B (69-60%)	C (59-50%)	D (49-40%)	
					
Information
Content
	Factually correct; as complete as could be expected in the circumstances.	No factual inaccuracies; extensive coverage.	Minor inaccuracy/ies but essentially correct; some important aspect(s) not covered.	Significant factual inaccuracy/ies; major omissions.	

Referencing
	Footnotes, references supplied where necessary; extensive bibliography.	Footnotes, bibliography extensive but not complete.	Footnotes, bibliography give only limited information.	Minimal/token footnotes and bibliography.	

Argument
and
Structure
	Information clearly structured; an argument emerges; critical treatment; essay coherent, cogent.	Information well structured; ideas developed intelligently; essay generally coherent.	Information could be structured more clearly; occasional breaks in structure.  Generally not very coherent.	Information poorly structured; difficult to follow logical connections; lacks coherence.	

Accuracy
of
Language
	(Virtually) error-free.	A few minor errors which do not distract reader from the content of the essay.	Several, intrusive errors which begin to distract attention from the content; most spelling rules followed.	Frequent, intrusive errors of basic nature disrupt the essay/alienate the reader; spelling deficient in places.	

Sophistication
of
Language	Level of language attempted complex; sophisticated command of lexis, syntax, and register.	Shows extensive appreciation of register and sophistication in lexis and syntax.	Shows some appreciation of register; mainly  simple lexis,  syntax; occasional  sophistication.	Predominantly simple in syntax and lexis; little awareness of register.	

* The use of unattributed material as if it is the student’s own intellectual property may result in failure of the essay as a whole and a mark of zero.

Word Count:  The mark will be reduced by 5% for every 10% by which an essay falls short of a stipulated minimum length.  Excess length may not be marked.

											Overall Mark:_______________________

B.  Criteria for Assessing Student Video Presentations

University of Birmingham, Department of German Studies

Name:________________________________

	A (100-70%)	B (69-60%)	C (59-50%)	D (49-40%)	
					
Pronunciation and
Intonation
	Accurate, varied; no noticeable English patterns.	Accurate, varied; very few English patterns.	Acceptable. Some flaws, but attempts to vary patterns; little  strain on listener.	Flat intonation; poor pronunciation; listener works hard to follow.	
Quality
of
Language	Minimal formal errors; wide range of structures.	A few formal errors do not distract from content; good range of structures.	Several formal errors but rarely interfere with communication; some attempt to vary structures.	Frequent formal errors; communication disrupted; little structural variety.	

Quality
of
Performance*	Excellent use of paralinguistic features/visual aids; completely at ease.	Good use of paralinguistic features/ visual aids; easy to follow.	Paralinguistic features enhance overall performance; heavy reliance on notes, but some audience awareness.	Little awareness of audience; poor paralinguistic features/use of visual aids. Whole sections read out, pace inappropriate.	

Content

	Comprehensive coverage; entirely relevant; cogent.	Wide coverage; most points covered in depth; relevant examples; well structured.	Reasonable coverage; relevant; some depth to points made.	Information largely correct and relevant, but superficial.	

Independence
	Exploitation of sources shows extensive ability to reformulate, summarize and expand.	Some material  effectively adapted and integrated. Some ability to reformulate, summarize and expand.	Content and language show basic evidence of independent exploitation.	Content and/or language
heavily indebted to sources but some attempt made to adapt them. 	

* Paralinguistic features include: body language, gestures, facial expressions and eye contact.

	Overall Mark:_______________________

Appendix 3

A.  Criteria for Speaking Tasks

The Open University

Comprehension (content 1): showing understanding of the stimulus material

20-25	All or almost all of the main points successfully communicated.	
15-19	At least three-quarters of the main points successfully communicated.	
10-14	At least half of the main points successfully communicated.	
5-9	At least a third of the main points successfully communicated.	
0-4	Less than a third of the main points successfully communicated.	

Organization and personal input (content 2)

(a)	structure (organization and presentation of material)

(b)	development and originality of ideas (extending the theme, expressing opinions and making judgements)

20-25	(a)  Very good organization (progression and coherence) of the                                                      material.
(b)  Very good development of ideas, personal input and/or opinions on the theme where required.  Very good level of analysis and conclusions, using relevant examples from the materials and/or elsewhere.	
15-19	(a)  Good organization (progression and coherence) of the material.
(b)  Good development of ideas, personal input and/or opinions on the theme where required.  Good level of analysis and conclusions, using relevant examples from the materials and/or elsewhere.	
10-14	(a)  Adequate organization  (progression and coherence) of the material.
(b)  Adequate development of ideas, personal input and/or opinions on the theme where required.  Adequate analysis and conclusions, using relevant examples from the materials and/or elsewhere.	
5-9	(a)  Limited organization (progression and coherence) of the material.
(b)  Insufficient development of ideas, personal input and/or opinions, analysis and conclusions.	
0-4	(a)  Very little evidence of organization (progression and coherence).
(b)  Ideas hardly developed and/or no personal input, opinion, analysis or conclusions expressed.	

Quality of language

(a)	accuracy

(b)	range of expression
20-25	(a)  Nearly always appropriate complex structures (e.g. tenses, subordinate clauses) with few errors.
(b)  Very good range and variety of vocabulary, expressions and complex structures appropriate to the task. Good use of idiomatic language.	
15-19	(a)  Appropriate complex structures (e.g. tenses, subordinate clauses) mostly accurate.  Simple structures usually accurate.
(b)  Good range and variety of vocabulary, expressions and complex structures appropriate to the task.	
10-14	(a)  Adequate use of simple and complex structures (e.g. tenses, subordinate clauses, agreements) but with some inaccuracies that do not generally impede comprehension.
(b)  Satisfactory range and variety of vocabulary, expressions and complex structures appropriate to the task.	
5-9	(a)  Frequent basic errors (e.g. tenses, subordinate clauses, agreements) 	that may hinder comprehension.
(b)  Limited range and variety of vocabulary, expressions and complex structures appropriate to the task.	
0-4	(a)  Frequent basic errors that hinder comprehension. Below acceptable standard for this course.
(b)  Range and variety of vocabulary, expressions and complex structures insufficient at this level.	

Pronunciation and fluency

20-25	Very good pronunciation and intonation. Communication is clear and unimpeded, except for minor errors. Flows very well.	
15-19	Good pronunciation and intonation. Communication is generally clear and unimpeded although there are some errors. Flows well.	
10-14	Acceptable pronunciation and intonation. Communication can at times be impeded. Reasonable flow.	
5-9	Inadequate pronunciation with errors that impede communication.  Little success at appropriate intonation. Does not flow well.	
0-4	Not enough language produced to assess pronunciation and fluency, or pronunciation such that it is (almost) impossible to understand.	

(Reproduced with kind permission of The Open University)

B.  Criteria for Written Work

The Open University

Comprehension (content 1): showing understanding of the stimulus material

20-25	All or almost all of the main points successfully communicated.	
15-19	At least three-quarters of the main points successfully communicated.	
10-14	At least half of the main points successfully communicated.	
5-9	At least a third of the main points successfully communicated.	
0-4	Less than a third of the main points successfully communicated.	

Organization and personal input (content 2)

(a)  structure (organization and presentation of material)

(b)  development and originality of ideas (extending the theme, expressing opinions and making judgements)

20-25	(a)  Very good organization (progression and coherence) of the material.
(b)  Very good development of ideas, personal input and/or opinions on the theme where required. Very good level of analysis and 	conclusions, using relevant examples from the materials and/or elsewhere. 	
15-19	(a)  Good organization (progression and coherence) of the material.
(b)  Good development of ideas, personal input and/or opinions on the theme where required. Good level of analysis and conclusions, using relevant examples from the materials and/or elsewhere.	
10-14	(a)  Adequate organization (progression and coherence) of the material.
(b)  Adequate development of ideas, personal input and/or opinions on the theme where required. Adequate analysis and conclusions, using relevant examples from the materials and/or elsewhere.	
5-9	(a)  Limited organization (progression and coherence) of the material.
(b)  Insufficient development of ideas, personal input and/or opinions, analysis and conclusions.	
0-4	(a)  Very little evidence of organization (progression and coherence).
(b)  Ideas hardly developed and/or no personal input, opinion, analysis or conclusions expressed.	

Accuracy

20-25	Appropriate complex structures and other elements of the language (e.g. tenses, subordinate clauses, agreements, spelling) with few or no errors.	
15-19	Appropriate complex structures (e.g. tenses, subordinate clauses) accurate in at least two thirds of the written work. Simple structures and other elements of the language (e.g. spelling, agreements) accurate in almost all cases.	
10-14	Appropriate complex structures (e.g. tenses, subordinate clauses) attempted and accurate in about half the cases. Simple structures and other elements of the language (e.g. spelling, agreements) accurate in two thirds of the written work, except for minor errors.	
5-9	Few successful attempts at appropriate complex structures (e.g. tenses, subordinate clauses). Frequent basic errors in simple structures (e.g. verb forms, genders) and other elements of the language (e.g. spelling, agreements).	
0-4	Frequent basic errors that hinder comprehension. Below acceptable standard for this course.	

Range of expression

20-25	Very good range and variety of vocabulary, expressions and complex structures appropriate to the task. Good use of idiomatic language.	
15-19	Good range and variety of vocabulary, expressions and complex structures appropriate to the task.	
10-14	Satisfactory range and variety of vocabulary, expressions and complex structures appropriate to the task.	
5-9	Limited range and variety of vocabulary, expressions and complex structures appropriate to the task.	
0-4	Range and variety of vocabulary, expressions and complex structures insufficient at this level.	

(Reproduced with kind permission of The Open University)

Appendix 4

Error Classification Sheet for German

I use this with post-‘A’ level students. The explanations in the right-hand box have been translated into English here for non-German speakers. Students receive this at the start of the module in the course handout.

A	Wrong expression, eg collocation problem:
Sie haben eine Entscheidung gemacht; should be  
Sie haben eine Entscheidung getroffen)	
Art	Misuse of article:
Er spielt die Geige; should be 
Er spielt Geige	
Gen	Wrong gender:
Das Bundestag ist das wichtigste Organ; should be
Der Bundestag	
K	Wrong case:
Aus verschiedene Gründe; should be
Aus verschiedenen Gründen	
Konj	Mistake with conjunction:
Sie hat viel Geld, trotz will sie es nicht kaufen; should be trotzdem	
MV	Modal verb error:
Laut dem Gesetz kann der Präsident den Vorschlag ablehnen; should be darf	
P	Misuse of pronoun:
Bevor dem Frühstück, für diesen Grund; should be
Vor dem Frühstück, aus diesem Grund	
R	Spelling error:
Selbsverständlich, compliziert; should be
Selbstverständlich, kompliziert	
T	Wrong tense:
Bevor ich gehe, habe ich ihn angerufen; should be 
Bevor ich gegangen bin, ……	
WS	Word order problem:
Sofort er ist verschwunden; should be
Sofort ist er verschwunden	
Z	Punctuation error:
Wir wissen doch dass er nächsten Monat kommt
Wir wissen doch, dass er……….	

NB also the following symbols that I will use in marking your work:

______  	major error
----------		less serious error
……….  	unidiomatic style
  ⋀		something has been omitted
   		re-order
(    )		omit/not needed
  ≠		repeat error

13.9  References

Alderson, C J and Wall, D (1993) Does washback exist?, Applied Linguistics 14, 115–29
Atkinson, T (2002) Computer Aided Assessment (CAA) and language learning [online]. ICT4LT Module 4 at: http://www.ict4lt.org/en [Accessed August 2002] 
Bailey, K M (1998) Learning About Language Assessment: Dilemmas, decisions and directions, Heinle and Heinle, Boston
Biggs, J  (1999) Teaching for Quality Learning at University, Society for Research into Higher Education and Open University Press, Buckingham 
Boud, D (1995) Enhancing Leaning Through Self-Assessment, Kogan Page, London
Brew, A (1999) Towards autonomous assessment: Using self-assessment and peer-assessment, Assessment Matters in Higher Education: Choosing and using diverse approaches, ed S Brown and A Glasner, pp 159–71, Open University Press, Buckingham   
Brown, G (2001) Assessment: A guide for lecturers, LTSN Generic Centre Assessment Series No 3, LTSN, York
Brown, G, Bull, J and Pendlebury, M (1997) Assessing Student Learning in Higher Education, Routledge, London
Corder, S P (1967) The significance of learners’ errors, International Review of Applied Linguistics, 5, 161–69
Council of Europe (2001) A Common European Framework of Reference for Learning, Teaching and Assessment, Cambridge University Press, Cambridge
Culwin, F and Lancaster, T (2001) Plagiarism, prevention, deterrence and detection [online]. Available for ILT members from: http://www.ilt.ac.uk/resources/Culwin-Lancaster.htm [Accessed August 2002] 
Dam, L and Legenhausen, L (1999) Language acquisition in an autonomous learning environment: learners’ self-evaluations and external assessments compared, in Learner Autonomy in Language Learning: Defining the field and effecting change, ed S Cotterall and D Crabbe, pp 89–98, Peter Lang, Bern/Frankfurt 
DIALANG (2002) http://www.dialang.org [Accessed August 2002]  
Ellis, R (1997) Second Language Acquisition Research and Language Teaching,  Oxford University Press, Oxford
Elton, L (1998) Are degree standards going up, down or sideways?, Studies in Higher Education 23, 1, 35–42
Gibbs, G (1992) Improving the Quality of Student Learning, Technical and Educational Services Ltd, Bristol
Graham, S (1997) Effective Language Learning, Multilingual Matters, Clevedon
Gray, C (1999) In defence of the secondary teacher? A PGCE tutor’s reaction to the great grammar debate, Language Learning Journal, 19, 40–45
Hughes, A (1989) Testing for Language Teachers, Cambridge University Press, Cambridge
James, C (1998) Errors in Language Learning and Use, Longman, London
JISC (2001) Electronic plagiarism detection project [online]. Available at: http://www.jisc.ac.uk/plagiarism/  [Accessed August 2002]  
Klapper, J (1997) Language learning at school and university: the great grammar debate continues (I), Language Learning Journal, 16, 22–27
Klapper, J (1998) Language learning at school and university: the great grammar debate continues (II), Language Learning Journal, 18, 22–28
Kleppin, K (1998), Fehler und Fehlerkorrektur. Fernstudienprojekt ,Deutsch als Fremdsprache und Germanistik‘. Fernstudieneinheit 19, Langenscheidt, Berlin/Munich
Kleppin, K and Königs, F (1991) Der Korrektur auf der Spur – Untersuchungen zum mündlichen Korrekturverhalten von Fremdsprachenlehrern, Brockmeyer, Bochum
Knight, P (2001) A Briefing on Key Concepts. Formative and summative, criterion- and norm-referenced assessment LTSN Generic Centre Assessment Series No 7, LTSN, York
McCulloch, D (1995) Where has all the grammar gone? An ‘accusative’ search, German Teaching, 12, 13–18
Powell, B (2001a) Understanding errors and mistakes, in Supporting Lifelong Language Learning: Theoretical and practical approaches, ed L Arthur and S Hurd, pp 139–51, CILT/The Open University, London
Powell, B (2001b) Assessment procedures and quality issues, in Supporting Lifelong Language Learning: Theoretical and practical approaches, ed L Arthur and S Hurd, pp 177–85, CILT/The Open University, London
Race, S (2001a) The Lecturer’s Toolkit: A practical guide to learning, teaching and assessment, 2nd edition, Kogan Page, London
Race, S (2001b) A Briefing on Self-, Peer and Group Assessment, LTSN Generic Centre Assessment Series No 9, LTSN, York
Schmidt, M and O’Dochartaigh, P (2001) Assessment in Modern Languages: Getting it right, University of Ulster, Coleraine
Selinker, L (1972) Interlanguage, International Review of Applied Linguistics, 10: 209–31
Sheppard, R (1993) Getting down to brass syntax: German teaching and the Great Standards Debate, German Teaching, 8, 2–9
Stefani, L and Carroll, J (2001) A Briefing on Plagiarism, LTSN Generic Centre Assessment Series No 10, LTSN, York
Tarone, E (1988) Variation in Interlanguage, Edward Arnold, London
Van Bremmer, S (1995), What is plagiarism? [online], Widener University, Chester PA, USA  http://science.widener.edu/svb/essay/plagiar.html [Accessed August 2002] 
Wakeford, R (1999) Principles of assessment, in A Handbook for Teaching and Learning in Higher Education, ed H Fry, S Ketteridge and S Marshall, pp 58–69, Kogan Page, London

13.10  Recommended further reading

Bailey, K M (1998) Learning About Language Assessment:  Dilemmas, decisions and directions, Heinle and Heinle, Boston
One of the most accessible introductions to language assessment theory. It includes good sections on test purposes, validity and reliability, norm- and criterion-referencing and approaches to marking. 
Although written for a specific American audience, it is readily accessible to all.

James, C (1998) Errors in Language Learning and Use, Longman, London
The most useful single volume for anyone interested in pursuing the question of error and error analysis. Rather specialist in places, but the sections on understanding learner error and its causes are worthwhile reading for all language teachers.
 
LTSN Generic Centre (2001) Assessment Series, LTSN, York
This set of four guides and eight briefings is aimed at HE staff and provides succinct, very readable overviews of important generic issues and practices in the field of assessment. The guides are intended for: Senior Managers, Heads of Department, Lecturers, and Students; while the briefings cover the following topics: Assessment Issues Arising from Key Skills, Assessment of Portfolios, Key Concepts in Assessment, Assessing Disabled Students, Self-, Peer- and Group-Assessment, Plagiarism, Work-based Learning, and Assessment of Large Groups. The Guide for Lecturers and the briefings on Key Concepts, Formative and Summative Assessment, Criterion- and Norm-referenced Assessment and Self-, Peer- and Group-Assessment are especially recommended. 

McNamara, T (2000) Language Testing, Oxford University Press, Oxford
A brief overview of the subject for the non-specialist which includes short surveys of different aspects of testing (uncluttered by references), readings from key texts and a selection of annotated references.

Race, S (2001) The Lecturer’s Toolkit: A practical guide to learning, teaching and assessment, 2nd edition, pp 31–103, Kogan Page, London
Chapter Two on assessment and feedback is an invaluable introduction to the subject. If you have time to read only one chapter on generic issues in assessment, make it this one. It includes numerous practical tips and suggestions.

Schmidt, M and O’Dochartaigh, P (2001) Assessment in Modern Languages: Getting it right, University of Ulster, Coleraine
A report on the 1998–2000 FDTL (Fund for the Development of Teaching and Learning) project on ‘Effective Practices in Assessment in the Modern Languages’. Based on a series of interviews with staff and students, and on QAA and institutions’ course documentation, this survey covers a wide range of generic and language-specific issues in assessment. It is linked to a ‘Workshop Handbook’, which aims to facilitate staff development workshops on assessment in languages. 

The ILT website (Members’ Area) contains the following papers on generic aspects of assessment, all with a wealth of references and links:

Baume, D  Portfolios for learning and assessment

Bostock, S  Student peer assessment

Johnson, R  Digest of online resources relating to assessment

Rust, C  Basic assessment issues and terminology

13.11  Assessment Task

Write a critique of the assessment scheme of a language module you teach:

1.	Describe the assessment procedures (coursework versus exams) and provide an outline of specific tasks. 

2.	How do the module’s learning outcomes link up with the assessment tasks? 

3.	How do the individual tasks measure up against the five key criteria of validity, reliability, transparency, practicality and washback on teaching?

4.	What potential is there for introducing into the module elements of student self-assessment, peer assessment, group assessment or CAA?

5.	Include a copy of the marking criteria used on the course and evaluate their effectiveness/appropriateness.

Annotated version of Figure 13.1 (section 13.5.2.1)

John Klapper:
Passiv: wird mit ‘werden’ gebildet: ‘ich werde gesehen’, 
‘ich wurde gesehen’, usw.

Mein Sommerurlaub 

Mein Freund und ich waren diesen Sommer in Koln Köln und wir hatten ein unangenehmes Erlebnis, denn wir waren von Taschendiebe Taschendieben am Bahnhof beraubt.
	Wir waren gerade nach langer müder ermüdender Fahrt aus dem Zug gestiegen  ausgestiegen und mein Bruder, der die Koffer tragte  trug, folgte mir. Aber weil wir den Bahnhof nicht kennten kannten, stellten wir die Koffer unten ab, um uns in an der Information nach einem Hotel zu erkunden erkundigen. Mein Bruder setzte setzte sich auf eine Bank und las die Zeitung, während ich zum Informationsbüro ging……..

Figure 13.1  Example of on-screen marking and feedback

John Klapper:
Starkes Verb: tragen, trägt, trug, hat getragen

John Klapper:
Ein Mensch kann müde sein, aber eine Reise ist ermüdend