Module 14

Varieties of Language Testing

This module is divided into the following sections:

Outcomes

Author of this module 

14.0   Introduction

14.1    Approaches to language testing
14.1.1	Purposes of language testing
14.1.1.1  Aptitude tests
14.1.1.2  Placement tests
14.1.1.3  Diagnostic tests
14.1.1.4  Progress tests
14.1.1.5  Achievement tests
14.1.1.6  Proficiency tests
14.1.1.7  Admissions tests
14.1.2	Direct and indirect testing 
14.1.3	Discrete-point tests versus integrative tests
14.1.4	Varying the approach to testing 

14.2  Assessing writing
14.2.1	Holistic marking
14.2.2	Analytic marking 
14.2.3	Objective marking
14.2.3.1  Model A
14.2.3.2  Model B
14.2.3.3  Model C
14.2.3.4  Model D
14.2.4	Essays
14.2.4.1  Positive features
14.2.4.2  Negative features
14.2.4.3  Improving the use of essays
14.2.4.4  Setting appropriate titles
14.2.5	Marks on scripts

14.3  Assessing speaking 
14.3.1	Role plays
14.3.2	Presentations
14.3.3	Oral exams
14.3.4	Student–student discussion
14.3.5	Other oral test types
14.3.5.1  Integrated skill tests
14.3.5.2  Picture stimuli
14.3.5.3  Information-gap activities
14.3.5.4  Explaining/providing instructions
14.3.5.5  Reading texts out loud

14.4  Assessing listening and reading
14.4.1  Listening
14.4.1.1  Task types
	14.4.1.2  Listening tasks  
14.4.2  Reading
		14.4.2.1  Reading tasks
14.4.3  Further ways to assess comprehension 
		14.4.3.1  Short answer questions
		14.4.3.2  Written summaries
		14.4.3.3  Free recall tests
14.4.4	Target-language testing
14.4.5	Multiple-choice questions

 14.5  Other forms of testing 
14.5.1	Using cloze and C-tests
14.5.1.1  Cloze tests
14.5.1.2  C-tests
14.5.2	Assessing translation
14.5.3	Using grammar tests
14.5.4	Projects
14.5.5	Portfolios
14.5.6	Reviews

14.6  Conclusion

14.7  Commentary on activities

14.8  Appendices
Appendix 1 	Criteria for assessing short essays
Appendix 2  	Criteria for assessing student oral presentations 
Appendix 3	Marking criteria for written work

14.9  References

14.10  Recommended further reading

14.11  Assessment task

ACCREDITATION INFORMATION
Those intending to count work on this module towards their university’s postgraduate programme, certificate or diploma in teaching and learning, or equivalent, are advised that sections 14.1–14.4 with the addition, where appropriate, of 14.5.2–14.5.3, along with the assessment task in 14.11, are to be considered core material, while remaining sections are extension material.

Outcomes

By the end of this module, you should be able to:

•	distinguish the different purposes of language testing;
•	use a variety of test types, including discrete-point, single- and mixed skill;
•	identify the pros and cons of analytic and objective approaches to marking;
•	improve your use of essays as an assessment instrument;
•	employ a variety of oral testing techniques;
•	select appropriate task types for the assessment of  the skills of listening and reading;
•	demonstrate awareness of the key issues surrounding target-language testing;
•	design cloze and C-tests;
•	assess translation as a communicative activity and develop suitable  marking schemes.

Author of this module
John Klapper is Director of the Centre for Modern Languages and Professor of Foreign Language Pedagogy, University of Birmingham. He is a trained teacher, a member of the ILT, a HEFCE National Teaching Fellow and from 1997–2001 co-directed the FDTL-funded DOPLA project. His current research interests are language teacher education, factors predictive of success in foreign language learning and the development of undergraduate language competence. He has published on different aspects of the learning and teaching of foreign languages, including teaching methodology, content-based learning, L2 reading, and has produced various commercial materials for the teaching and learning of German and Russian. 

Contact: j.m.klapper@bham.ac.uk

14.0 Introduction

The vast amount that has been written about assessment testifies to its  crucial role in promoting effective learning and teaching. Broader assessment issues (such as the role of learning outcomes, formative and summative assessment, norm-/criterion-referencing, and feedback) are the subject of Module 13. This module is concerned exclusively with language-specific aspects of assessment, in particular the science of language testing. This is itself an extensive field and so all we can hope to provide here is an introduction to the subject, focusing on its relevance for the sort of testing carried out in university foreign language (FL) departments and language centres.
As linguists, we are likely to find ourselves wondering at various times what the best method of testing students is. The fact of the matter is that there is no single ‘best’ test or method. Each assessment situation is different and will require a different approach to testing, dependent on the learning outcomes, the purpose, status and importance of testing, and on the time and resources (physical and human) that are available. It is the aim of this module to help language tutors make informed decisions on these matters.
Before we begin, a word about test classification. Language teaching takes a variety of forms in higher education and testing appears in a similar variety of guises:
1.	Formal, ‘sit-down’ examinations; students are given results but no other feedback on performance and they do not get to see their scripts (eg end-of-year and finals examinations).
2.	Semi-formal tests, taken under test conditions, with marks possibly contributing to course assessment; students get their papers back and discuss them with the tutor, or else receive advice on their oral performance.
3.	Informal tests, carried out in class at regular intervals; results do not ‘count’ for anything or contribute towards a final grade; they are designed primarily to provide short-term motivation.
In a university department or school, teaching ‘specialist’ students of modern languages, type (1) may be the norm. In a university language centre operating an institution-wide language programme, type (2) might be more common, although (1) could also feature here. On some ‘leisure’ or adult education language courses, on the other hand, (3) would probably be the norm, with  occasional use of (2) on accredited courses. The principles and practices outlined in this module apply primarily to types (1) and (2), although many of the techniques for testing the various language skills can readily be employed for type (3) assessment too. 

There are clear links between this module and Module 13, indeed the two are in many ways complementary. Ideally, you will have looked at the earlier module before starting work on this one. At the very least, you are strongly advised, before tackling the material here, to read section 13.1.4 on ‘Key features of assessment’. 

14.1  Approaches to language testing
This section provides a general introduction to the field of language testing. First we examine the various reasons for testing a student’s language. There then follows a discussion of some major language test classifications, including the difference between direct and indirect testing, and the use of discrete-point tests versus integrative tests. Finally, we consider the importance of employing a range of language testing techniques.

14.1.1  Purposes of language testing

Activity 1
Decide which of the following statements apply to your department: 

In my department, testing is used……
	Yes	No	Not sure	
1.	as a selection process.
				
2.  to help the students learn material more 
     effectively.				
3.	to increase motivation.
				
4.  to help the tutor discover what the students
     do not know or cannot do.				
5.  to show students what they can do.
				
6.  to provide evidence of whether students 
     have progressed.				

Language testing can be carried out for a variety of purposes, and while in higher education we are only likely to be interested in three or four of them, the following summary is included to help clarify precisely why we are assessing our students and what is involved in each case.

 14.1.1.1  Aptitude tests
These are not related to any one language but assess a student’s capacity for learning any language. Language aptitude is a complex (and controversial) topic. Elements of aptitude testing are found in 11+  and some intelligence tests. There are commercially produced tests available and good discussions of issues relating to aptitude can be found in Spolsky (1995) and Skehan (1998).

14.1.1.2  Placement tests
These are sometimes confused with diagnostic tests and indeed they often serve both purposes. Strictly speaking, however, a placement test is designed to identify which level in a multi-level programme a student should be allocated to. Normally one would expect the test material to be related to the curriculum or syllabus of the various levels. For example, the University of Birmingham’s Centre for Modern Languages has a Web-based multiple-choice placement test for its Open Access programme which enables students to find which of eight possible levels they should enrol on.

14.1.1.3  Diagnostic tests
These tests should be a more finely-tuned instrument than placement tests, although, as noted above, for pragmatic and practical reasons, they are often merged. Their purpose is to identify strengths and, in particular, weaknesses in the FL, so that tutors can shape the syllabus to take account of what the diagnosis reveals, students can take remedial action and individual programmes of work can be designed.

14.1.1.4  Progress tests
As a course develops, tutors need to have information on how students are coping with the course and how they are developing linguistically. Coursework on most modules will include some such check on progress, possibly in the form of informal recap exercises or quizzes, possibly in the shape of more formal class tests at the end of each unit of work.

14.1.1.5  Achievement tests
These are the most common tests found in university language programmes. They usually occur at the end of a module or year of study and aim to assess how well students have assimilated course material and whether they have achieved the learning outcomes of the course. In the later stages of a degree programme, as students’ linguistic proficiency becomes more varied and advanced, such achievement tests can take on the appearance of proficiency tests (see below) which strictly speaking is inappropriate for anything other than a summative final-year exam intended to attest to a student’s overall linguistic proficiency. In other circumstances (and some would argue even in final year – see Module 13, section 13.1.3), it makes little educational sense to test students on substantial amounts of material that have not been covered in the course syllabus.

14.1.1.6  Proficiency tests
Unlike a test of achievement on a particular course, a proficiency test aims to assess all-round ability in a language in varied contexts, independently of the setting (naturalistic or classroom-based) in which the language was acquired. Examples in the UK are the Institute of Linguists’ exams or the Diplomatic Service Language Allowance exams which accredit language skills at a number of levels of proficiency. Such tests are notoriously difficult to set. Although a finals exam is probably the closest university language programmes get to such professional tests, there is still a clear difference between the two, given the bias towards translation and ‘academic’ writing tasks in the final year of many language degree courses.

14.1.1.7  Admissions tests
For the sake of completeness, we should include these tests. They aim to ascertain whether a student will be able to cope with the demands of a particular course or programme of study. Since modern language departments rely on grades in public exams for entry to degrees, admissions tests are only ever informal affairs, used when determining the ability of entrants with 'non-standard’ qualifications. However, TOEFL (Test of English as a Foreign Language) and IELTS (International English Language Testing Service) scores act as forms of admissions test to most universities for non-native speakers of English.

Activity 2
By way of reviewing the above, try to complete the following table (if you like, as a form of progress test!). Insert one of the following against each of the test descriptions in the box below:
•	
•	Admissions tests
•	Achievement tests
•	Aptitude tests
•	Diagnostic tests
•	Placement tests
•	Proficiency tests
•	Progress tests

Test description	Test type	
These assign students to appropriate levels within a programme; such tests are related to course objectives.		
These assess students’ general ability to learn languages prior to actual tuition in any particular language.		
These are used for screening purposes to make decisions about students’ ability to cope with a course.		
These assess FL speakers’ ability in the target language independently of any curriculum.		
These are used during a course to provide information about students’ mastery of or problems with elements of the syllabus.		
These identify students’ existing strengths and weaknesses in order to help tailor tuition to learners’ needs.		
These assess what students have accomplished in relation to a particular course of study or module. They usually occur at the end of a course.		

(Adapted from Bailey, 1998: 40)

Click on ‘Commentary’ to check your responses. 

14.1.2  Direct and indirect testing
An important distinction in language testing is made between ‘direct’ and ‘indirect’ tests, that is between those that involve learners in actually performing the skill that is being assessed (direct) and those that assess the  language elements which are thought to support or underpin the four main language skills (indirect). 
	The first theories of language testing were based on structuralist linguistics (Lado, 1961) and assumed that to know a language meant no more than having a mastery over aspects of the language system. Tests therefore tended to separate out these aspects and test candidates’ knowledge of discrete areas such as grammar, without much surrounding linguistic context. It was felt that indirect testing was the only reliable and consistent means to measure language ability. Examples of such tests include: 

•	testing discrete grammatical items as an indirect test of FL writing, 
in the belief that knowing grammatical forms is fundamental to 
writing in the FL;
•	testing the ability to recognize FL sounds or imitate intonation 
patterns as an indirect test of FL speaking, since pronunciation and 
intonation are key elements of being able to speak;
•	testing knowledge of vocabulary as an indirect test of FL reading, 
since lexical knowledge is a key component of being able to read;
•	testing the ability to fill in the gaps in a transcription of an FL 
conversation (known as a conversational cloze test) as a way of 
indirectly testing speaking ability (see Hughes, 1981).

	The advantage of indirect tests is that they are thought to be a more precise and objective method of assessment, allowing linguistic features to be tested discretely or separately, and ensuring assessment is transparent and highly reliable: answers are either right or wrong and markers do not need to apply much judgement. Moreover, such tests are simple to administer and are thus very practical. Consider the following: 

Gestern ist sie mit meinem Bruder in _____ Hallenbad gegangen.
 	(a) dem		(b) das

The student here simply has to ring the correct word (accusative or dative direct article) to go in the gap; there is no writing involved and only minimal reading. 
However, such test items are not without their problems. In the above example, with only two possible answers, there is potential for guessing, so one might extend the question as follows:

Gestern ist sie mit meinem Bruder in _____ Hallenbad  gegangen.
(a) dem		(b) die		(c) den 	(d) das 

Although this reduces considerably the scope for guessing, it introduces a further complication in the shape of the gender of the noun so that the student does not just have a simple distinction to make between the use of the accusative or dative case, but now also has to decide whether the noun is masculine, neuter or feminine. This is a simple illustration of the potential disadvantages of such indirect tests. It can, in fact, be very difficult to produce a simple discrete test of a single linguistic point.
Another problem is that no one really knows if indirect tests are valid tests of the skills they claim to measure. Is someone with a good knowledge of FL vocabulary necessarily a good FL reader? And does being able to reproduce sounds and intonation necessarily mean you are a good FL speaker? Most tutors will have experience of students being able to spot errors in the FL when they are asked to identify them in exemplar material (a form of indirect testing) but then proceeding to make the same errors in their own FL writing. Experience suggests there is often a link between indirect tests and the skills they purport to assess, but one cannot say with real certainty that there is.
	Furthermore, there are implications for teaching in indirect tests. As Bailey (1998: 75) notes: ‘Indirect tests […] may result in negative washback. For instance, if learners spend time studying bits of decontextualized grammar in preparation for an indirect test of writing, they may spend less time actually writing in the target language.’ (See Module 13, section 13.1.4.5, for more on ‘negative washback’.) This is a real concern in programmes that focus excessively on grammar and translation, where the teaching of FL writing can be seriously neglected (see Module 9). 
Growing recognition of the fact that language performance involves the integration of these discrete aspects of linguistic knowledge led to the development of skills testing, ie of separate tests of, speaking, writing, listening and reading. This was given impetus by the spread of communicative language teaching and the need for tests that could measure integrated performance and assess the ability to apply knowledge of the linguistic system in the achievement of communicative goals. (For a fuller discussion of these developments, see McNamara, 2000: 13–21.)
Since language teaching in all areas of higher education has been influenced to some degree by elements of communicative methodology, the present module adopts this basic communicative approach to skills testing. However, it does not ignore indirect testing or the enduring use of translation testing, nor indeed more recent innovations in assessment  (see 14.5.2–14.5.6). For the sake of simplicity and clarity, we deal with each of the four skills in turn, but that is not intended to recommend this approach as some sort of norm: just as much language teaching nowadays features tasks involving more than one skill, so in assessment you may well want to employ integrative tests that assess students’ ability to use a combination of skills. 

14.1.3  Discrete-point tests versus integrative tests
An example of this ‘integrative’ testing would be a task requiring students to listen to a simple telephone message and to draft a memo in reply. This  involves the linguistic elements of phonology, grammar, vocabulary and discourse, as well as the skills of listening and writing. Or, again, a task requiring learners to read an e-mail in the FL and relay its contents orally to another person on the telephone, involves an equally wide range of linguistic elements, as well as the skills of reading and speaking. To refer back to our earlier distinction, discrete-point tests are normally indirect, while integrative tests are usually direct.

Activity 3

1.	See if you can think of two further language tasks that involve integration of skills. 

2.	Which linguistic elements are involved in your tasks?

One interesting variation on integrative tests was Oller’s (1979) development of cloze tests (see section 14.5.1.1) which, along with C-tests (see section 14.5.1.2), represent exceptions to the direct–integrative and the indirect–discrete alignment, being essentially indirect tests that integrate a range of language skills and linguistic elements. In accordance with the so-called ‘Unitary Competence Hypothesis’ (cf 14.5.1), Oller believed that ‘performance on a whole range of tests […] depended on the same underlying capacity in the learner – the ability to integrate grammatical, lexical, contextual and pragmatic knowledge in test performance’ (Oller, 1979). It was thought the cloze test was capable of measuring this competence. Although they are still quite widely used, it has been found, in fact, that cloze tests tend to measure much the same things as discrete tests of grammar and vocabulary. 
	Proponents of objective tests would argue that integrative tests can lead to unreliable marking since they frequently require a degree of subjective judgement on the part of the marker. However, detailed and thoroughly trialled criteria (see, for example, Appendices 1–3), allied to the standardizing procedures for teams of markers employed by organizations involved in large-scale testing, such as the secondary awarding bodies or The Open University, ensure a good degree of transparency and high reliability on direct tests involving speaking and writing. Student numbers in conventional university language assessment may not warrant such detailed procedures, but the same tried and tested criteria, thorough and agreed mark schemes, moderation of marks and/or second marking should ensure similar reliability.
	Ultimately, the choice of direct versus indirect testing will be a pedagogical one. As noted in Module 13, section 13.1.3, all assessment should be closely related to learning outcomes and should have the effect of a positive washback on classroom learning and teaching (see Module 13, section 13.1.4.5). As Bailey concludes (1998: 83): 

The crux of the matter for us as classroom teachers choosing between direct and indirect measures of a skill is to decide what we want to measure. Do we wish to focus very tightly on specific aspects of enabling skills? (This might be the case in a diagnostic test or a progress test.) If so, an indirect test might be useful. Or do we wish to assess our learners’ abilities to use the macroskills in the target language? In that case, a more direct test would be called for.
As this suggests, the apparent direct–indirect and integrative–discrete dichotomy is no such thing. The terms should rather be seen as denoting points on a spectrum: depending on its purpose, a language test will be located closer to one end or the other, while a single examination might well feature test elements from various points on the spectrum.
 
Activity 4

1.	Choose TWO of the four skills of listening, speaking, reading and writing, and for each of them think about how you would assess students’ ability through one direct and one indirect test. 

2.	Consider also how you would mark each of the resulting four tests.

14.1.4  Varying the approach to testing 
In addition to the pedagogical reasons for choosing a particular test type, tutors need to consider the value in employing a variety of testing techniques with any one group of students. There are several reasons for doing this:

•	No single test can be absolutely reliable; therefore the more varieties of testing that are employed, the less likelihood there is of bias or distortion affecting the assessment process.
•	Students are known to have different learning styles (see Module 12) and respond differently to styles of teaching and techniques of testing; only through variety can we make allowance for this. 
•	Different students excel in different areas of language. Traditionally, tests have focused on grammar and writing to the exclusion of other aspects of language, but not all students are good at grammar. More creative users of the language, in particular, can easily be discouraged by such a narrow focus, since the things they are good at are not tested. So the wider the range of areas covered by our testing, the fairer this will be to all.
•	Since tests are a harsh fact of life in all language learning, including even ‘leisure’ courses, employing a variety of informal and formal testing techniques helps to get students used to this reality from the start.

Activity 5
Consider assessment practices in your department, or in one you are familiar with either as tutor or student, and decide which of the following statements is most applicable:

In my department….	Yes	No	
1.  writing is favoured as the main means of testing 
     students.			
2.  students are tested regularly both formally and 
     informally.			
3.  assessments of particular skills feature more than 
     one test type. 			
4.  students are given opportunities to ‘shine’ in 
     speaking skills.			
5.  listening and reading are regularly tested.

14.2  Assessing writing
There are three main ways to mark students’ work, typically referred to as holistic, analytic and objective marking. These approaches are equally applicable to speaking and writing tasks, but in illustrating their use in this module we shall focus on the assessment of written work. (The assessment of speaking is the subject of section 14.3.) In this section, after looking at specific examples of the three approaches ‘in action’, we consider the pros and cons of FL essays and the most effective way to use them for the testing of writing. Finally, we look at the question of annotating students’ work and systems for penalizing linguistic shortcomings.

14.2.1  Holistic marking
This approach assesses students’ written product as a whole, without separate reference to such sub-components of writing as coherence, breadth of vocabulary, accuracy, structure, etc. Typically, a single mark is awarded to the piece of work, eg on a scale of 1 to 5, 1 to 10 or A to E, and each mark is linked to some written criteria describing the threshold level of performance the student needs to reach to achieve that mark. This acts as the sole guide to the marker. For example:

Grade 1  Shows clear competence in writing. Work awarded this grade:
- is well structured;
- is coherent throughout;
- addresses all aspects of the task;
- develops all its arguments well;
- displays a wide range of vocabulary;
	- employs complex syntax.

Grade 2  Shows competence in writing. Work awarded this grade:
- is generally well structured;
- is largely coherent;
- addresses the task, but handles some aspects more 
  effectively than others;
- develops its arguments well, but may feature fewer details 
  than Grade 1 work;
- displays a good range of vocabulary;
- shows a variety of syntax and is largely accurate.

Grade 3  Shows limited competence in writing. Work awarded this grade:
- is adequately structured;
- shows some coherence, but parts may be difficult to follow;
- omits to address some aspects of the task;
- employs some details to support its arguments;
- displays a satisfactory range of vocabulary, but may feature 
  some repetition; 
- shows inconsistency in use of syntax and features several 
  errors. 

Grade 4  Shows minimal competence in writing. Work awarded this grade:
- is inadequately structured;
- is difficult to follow and rarely fully coherent;
- only addresses a small part of the task;
- fails to support its arguments with much detail at all;
- employs a narrow range of vocabulary;
- demonstrates a very uncertain grasp of grammar and 
  features a very large number of errors.

Grade 5  Shows lack of competence in writing. Work awarded this grade:
- is very poorly organized, with no clear structure;
- is impossible to follow in some sections and generally lacks 
  coherence;
- does not address the task;
- includes little or no detail;
- employs an extremely limited vocabulary; 
- features frequent and serious errors of grammar; well over 
  half of the work is inaccurate.

There are both advantages and drawbacks to this type of approach. On the plus side, one can note that:
•	marking scale and standards are clear to all;
•	markers can achieve a high degree of comparability and reliability;
•	it tends to be positive in its approach to marking;
•	it lends itself to assessment of a variety of free writing tasks.
On the other hand:
•	broad categories may not differentiate between students sufficiently;
•	it does not provide much feedback to students at all;
•	the related criteria are general and do not allow for the demands of different writing tasks (eg discursive essay versus letter of complaint).

14.2.2  Analytic marking
In this approach, the sub-components of writing are analyzed and included as categories to be marked separately – eg Content, Structure, Accuracy,  Range of Language (see Appendix 1 for an example). Sometimes the sub-skills are equally weighted, sometimes they are differentially weighted (see Appendix 3). 
The advantages of the analytic approach are:

•	the assessment scheme is a direct reflection of the teacher’s 
understanding of the writing process;
•	it allows students to see clearly what they are to be assessed on    
and what marks will be allocated to each sub-skill;
•	in theory, such an approach, focusing attention on different sub-
categories and their criteria, will produce a more accurate and a   
fairer assessment of a student’s overall performance on the task; 
•	it provides the student with more effective feedback on the different 
aspects of the task; 
•	it can produce positive washback on teaching and learning.

The disadvantages of analytic marking are:

•	introducing several sets of criteria can make it more difficult to 
attain marker reliability, ie there are more descriptors to disagree 
about;
•	marking can take a lot of time if one has to assess a candidate 
under maybe five different headings;
•	in averaging marks awarded for different categories, the range of 
final marks awarded tends to be narrower than when impression 
marking is used;
•	criteria can only be written with the typical student in mind, but few 
students are typical;
•	criteria take a lot of time to devise and need to be revised in the 
light of experience;
•	it can sometimes be difficult to judge each of the categories 
independently of the others; a ‘halo effect’ can quickly develop 
when you are marking an able candidate, and the opposite can happen with a weak candidate. 

During the interviews on assessment conducted for Module 13 a student made the following comment on the use of marking criteria in his university:
I don’t think one half of our department really knows what the other half is doing. Different lecturers seem to have different ideas about marking, and although there is a grid for language work in the departmental handbook, when we do essays or an oral presentation, some don’t seem to use the grid, while others seem to be using different sorts of grids.
 Although it may not be practicable to devise different detailed criteria for every single language task, departments should have agreed criteria for the range of assessment tasks they most commonly employ. Typically, this might include oral presentations, oral interviews, FL essays, summaries and translations. Moreover, these criteria must be specific to language performance and should not be confused with general criteria for work in other areas of the curriculum, such as ‘intellectual rigour’, ‘critical thinking’, ‘powers of analysis’. 
Students should certainly be given copies of marking criteria in advance of doing the assessments, but it is just as important that they see all their tutors actually using them and doing so consistently. Furthermore, feedback ought to refer closely to them. You may like to consider whether students should be given exactly the same criteria as those used by tutors, since departmental criteria often feature needless metalanguage. 

Activity 6

Have a look at Appendices 1, 2 and 3 and compare them with each other. Are there any problems here with metalanguage? Which, if any,  would you prefer to give to your students?

Despite the importance of criteria, students need to realize that although criteria are there to provide a measure of objective guidance, a lot of assessment of FL writing remains subjective, and fine-grade decisions are a matter for academic judgement.

Activity 7

Imagine you have just arrived at a university in France to teach English as a foreign language. As one of your first tasks you are asked to mark a set of essays, such as the one below.

I have to say I am very ashamed be French at this moment here. This is because we have had a very poor election result and the representative of an far-right extreme political party, mister Jean-Marie Le Pen has won through to the second round of the presidential election race. I can not explain to myself why this has happened.

Probably the reason for this shameful thing that shows France in a bad light to the international community is that the voters in France at this time here are not interested in politics. Also however the President mister Chirac has to be blamed because a lot of people think he is corrupt and has not used the office of President properly.

It think it is a very bad thing that the socialist party has done so badly and I am angry that it’s supporters have not gone out to vote (maybe in protest) and as result the extremists have been permitted to gain such a big ‘victory’. 

I can hope and pray that in the next and final ballot the French people will realize the danger of the far-right extremists and will exit to vote in large number to stop Le Pen, even though the majority will not be supporters of Chirac. However there has been a lot of damage to France and its images abroad. It will take a long time to put this to right.

What would be your initial reaction to this request? What would you want to know? Why? After you have given this some thought, click on ‘Commentary’.

Activity 8
Again you are going to be put in the position of a marker. This time you are going to be given a bit more information to help you. First, read this job advert.

Is this you?
Sociable and outgoing? A native speaker of French, German, Italian or Spanish? Keen to spend the summer in one of Europe’s leading cities of culture? Looking to gain work experience in the tourist industry? 
If so, then read on…
The Scottish Tourist Board is looking for four student reps to organize and accompany visits to Glasgow and Edinburgh by groups of school pupils from mainland Europe during the period July to September.
Please apply by letter with accompanying CV to the address below, mentioning relevant experience and stating your reasons for wanting the job.

The Scottish Tourist Board
Coldstream House
Mews Walk
Edinburgh
E2 5PQ

Imagine you have to mark the two letters below written by overseas students in response to the advert. Write appropriate analytic marking criteria for this exercise, bearing in mind the demands of the task and assuming the students are the equivalent of first-year post-A level students of modern foreign languages in the second week of their course. You might like to use the criteria in Appendix 1 as a starting point and adapt them to this particular task. What are the relative strengths and weaknesses of the two letters? And which scores higher on your criteria?

Letter 1

Dear Sir/Madam,

I write about your advertisement in Guardian newspaper to find student rep for the summer works.

I am all those things you write about and in particular am get on well with all people I know. I work in supermarket on evenings after my study. I French and Spanish speak very good and little Portuguese. I have last summer worked hardly in tourist office in Barcelona and so having had expereince of such work.

I apply for this job becaus I love your country a lot since I went on holiday there with my family and because I want in the future to do work in tourist industry and this will give me abundant expereince. Also I like work with young people in all ages and size and I think I will make up a good group leader for young groups.

I send with this letter my CV. I hope you like my application and I look forward to hearing your reply.

Yours faithfully,

Letter 2

Dear Scottish Tourist Board

I would very much like to work for you as a holiday rep for groups of overseas students on visits to Scotland. I saw your ad in the paper on 14th April. I am sure my knowledge of German and French (fluent) as well as my conversational Dutch would be extremely helpful to you. I am very easy going and manage to get on with most peoples. Although I have never worked in Scotland and I understand the accent can take a little getting used to, I am sure I shall be fine. Look forward to hearing from you soon,

Best wishes,

Click on ‘Commentary for feedback on this task.

14.2.3  Objective marking
Objective assessment denotes a system of allocating marks according to a clearly quantifiable mark scheme. While a lot of listening and reading tests lend themselves to numerical marking (eg one mark for each correct element of an answer), speaking and writing tasks are notoriously difficult to quantify in such a clear, objective manner. That is why assessment schemes for these tasks usually rely on written criteria and descriptors of expected performance at different levels, against which students’ work is matched. Criticisms of this approach include, above all, the charge that it involves too much subjectivity, as individual tutors are likely to interpret expressions such as ‘acceptable level of fluency’, ‘a high degree of accuracy’, or ‘extensive coverage of the topic’ in their own way, based on their preconceptions of these terms and their expectations for work at a given level.  
	Some tutors consequently prefer to employ more objective approaches to marking the productive skills, especially free-writing tasks such as essays. 

Activity 9

Before reading any further, think about your own approach to marking essays or that of your department. How do you account for the number of errors in a piece of work? Or the use of structures? Are there any aspects of this approach you are not happy with?

There are a number of different approaches here but most involve some form of error quotient to assess accuracy and command of the language. I asked four tutors to describe their approach to objective marking of essays. Their responses are presented as the four models below.

14.2.3.1  Model A

This tutor works in a language department. She first counts the total number of words the student has written (time can be saved by getting students to do this themselves; alternatively a quick sample count of typical line length multiplied by the total number of lines can help verify the count). Next, she underlines all the errors (including spellings, syntax, morphology, vocabulary, coherence across clauses and sentences, connectors), and totals them. She then standardizes the word length, ie if a target essay length were 400–500 words, she might take 450 as her standard length and calculate how many errors the student would have made if he/she had actually written 450 words. For example, if the total word length is 500 and the total number of errors is 45, she reduces 45 by 10% (= 50 words off 500) to give a final score of 40.5. By dividing the standard length by the standard score (450/40.5) she obtains an error quotient. This approach ensures comparability across the group of students and means that those who write longer essays, and presumably therefore make more errors, are not penalized for doing so, while those who write less are not advantaged, compared to others, through the resultant reduction in their potential error count.

14.2.3.2  Model B

This tutor teaches on an institution-wide language programme (IWLP) Her approach also involves doing a word count but she only marks the first (or last) X number of words (eg 300). She underlines all the errors, allocating a mark to them according to how much they interrupt the fluency of the writing. If the focus of the task is grammar, she might weight them according to the significance of the formal error. For example, ½ for a minor error such as spelling or orthography that does not detract from the flow of the text; 1 for an error such as a case ending, adjective agreement or minor word order problem, that affects readability but not in a major way; and 1½ for an error that interrupts the flow of the text, makes the message unclear or ambiguous and thus confuses the reader, such as inappropriate tense, misleading pronominal reference, wrong choice of vocabulary or poor use of logical connector. The tutor adds the various scores (say, 40) and calculates an error quotient using a slightly different approach to that of the tutor in Model 1: she divides the error total by the number of words marked, thus removing the need for a calculation of standardized length, ie 40/300 = 0.13. This tutor does also mark other material beyond the 300 but excludes it from her calculations to ensure a degree of comparability.

14.2.3.3  Model C

This tutor works in a language department. His approach is similar to 1 and 2, but he penalizes errors with just a 1 or 2 and ignores all spelling and orthographical errors for marking purposes. He too does a word count but accepts students’ variable lengths and calculates an error quotient by dividing the total number of errors by the total word count (eg assuming a length of 459 words and 61 errors: 61/459). However, he applies a standard penalty of 5% off the final mark for any essay over the stipulated maximum or under the stipulated minimum. In addition to the ‘marks off for errors’, this tutor has a parallel scheme of bonus marks (+ ½ or + 1) for good use of complex structures or idiom.

14.2.3.4  Model D

This tutor works on an IWLP. He too calculates an error quotient by dividing the number of errors by the total number of words written but multiplies this by 100. So, if there are 20 errors and 200 words were written: 20/200 = 0.1 x 100 = 10. To simplify marking, the tutor does not distinguish between errors, counting them all as 1, does not penalize spelling mistakes or repeat errors. He also counts multiple errors in a complex construction as one mistake only.

These various approaches all have one thing in common: any score obtained via calculation of an error quotient still has to be converted into the local marking currency. For example, if final essay marks are expressed as marks out of 100, then a further calculation has to be performed to convert, say, 450/40.5 into a percentage.
	Apart from this complication, the tutors interviewed were aware of some disadvantages in their approach. These can be summarized as follows:
•	Calculating total errors is an essentially negative approach to marking and fails to acknowledge what students manage to get right. (The approach of the tutor in Model C provides a partial solution to this).
•	Focusing on errors, fails to award good organization, rich content or originality of ideas. Consequently, quantitative approaches need to be supplemented by qualitative judgements of these issues, using clearly expressed criteria.
•	Even apparently objective mark schemes can hide subjective elements. One tutor is likely to differ from another in his or her understanding of what ‘major’, ‘minor’, ‘significant’, ‘distorts message’, etc, actually mean when allocating marks to different types of error.

Activity 10

EITHER

Read the piece below written by a learner of English as a foreign language, an ethnic Korean raised in Japan.

This a very shameful experience to me.

When I visited to Korea first time in 1980.

I took a taxi from airport. It was rainny night and really hard to cought the taxi. Because there are so many people at the airport.

At last I caught the taxi after I waited about 40 minutes. The taxi was begin to run.

After drove about 5 minutes the driver went to other way that was made me feel some fear.

I asked him “why you going this way?”

He said “Oh, don’t worry this is a short cut!!”

After a little while he made stop a car and he said “I think there is some troble with engine.”

I was really scared because it was was really dark and became terrible rain. He said “I’m sorry but I need your help. Whoud you push my car?”

Then I did. I was of course leave all of my stuff in the car. At the time, I suspected what I going to happen from now.

I added all of my power and push the car.

Then he was gone.Yes. he steal all of my stuffs. And I never find my stuff again.

I was really sad, but I couldn’t tell any my Japanese friends. Because I am Korean.

Length: 215 words

(Quoted in Bailey 1998: 200)

Mark the work using each of the following three schemes in turn: 
a)  the holistic scheme in 14.2.1
b)	the analytic scheme in Appendix 1
c)	the objective approach outlined in 14.2.3.2.  
Express your mark as a percentage in each case. Did the marks you awarded differ significantly? Why do you think this is? Which approach did you feel happiest with?

OR

Ask two colleagues to do this exercise with you. Each of you should  employ one of the above mark schemes. Compare your results.

Activity 11
For this activity you will first need to read or remind yourself of the key criteria for evaluating assessment tasks. (See Module 13, section 13.1.4). Referring to the box below, decide how many of these criteria apply to each of the three approaches to marking essays.

	Holistic	Analytic	Objective	
Validity				
Reliability				
Transparency				
Practicality				
Washback				

Click on ‘Commentary’ for feedback on this task.

The three approaches to marking students’ written work discussed in this section differ considerably and, as Activity 11 shows, each has its  strengths and weaknesses. In the particular context of higher education, holistic marking is unlikely to prove sufficiently flexible or to discriminate student performance adequately for the purposes of learning and teaching. While analytic schemes are becoming widely accepted, many feel their lack of reliability in assessing students’ command of grammar and vocabulary require an element of objective scoring alongside descriptive criteria for other components of writing.

14.2.4  Essays
Having looked in depth at approaches to marking written work in the FL, we shall now consider briefly the pluses and minuses of that ubiquitous writing task in universities, the essay.

14.2.4.1 Positive features
Although FL writing is in all probability going to be the least used skill outside and beyond the university language programme, this is not a reason for neglecting it, either as a teaching activity or an assessment method. Free writing in the FL reveals very clearly students’ underlying weaknesses in the language, ones which in spoken communication might be labelled 'performance error' or put down to ‘communicative pressure'.
Moreover, essays or summary writing are an especially effective vehicle for assessing higher-level language skills, or cognitive academic language proficiency, since they allow students space for distinctive treatment of a complex topic and give them scope to show what they can do linguistically, ie to employ varied structures, a wide range of vocabulary and idiom.

14.2.4.2 Negative features
Against these arguments one might say that those who have mastered the art of essay writing in L1 and know how to structure an academic essay will have an advantage over others. This may not seem important for students of modern languages since they will in any case need to master essay-writing skills for non-language elements of their programme, but students of other disciplines who are studying a language as a supplementary element may well be at a disadvantage and need support in this area.
As we have seen in discussing marking criteria, essays are harder to mark fairly, however good the criteria might be. Objectivity can never be achieved; no matter how closely one follows criteria, it is difficult to avoid subjective judgements. Two colleagues using the same criteria to mark the same essay can sometimes come up with significantly different marks.
	Essays also take a lot more time to write and to mark. Consequently, their use can limit how much of a module is assessed; for example, on a typical language course, an essay might test students’ knowledge of the language relating to, at best, two or three topic areas.
	Furthermore, it is easy to allow our marking to be influenced unduly by a preceding essay of particularly high quality, thus raising our expectations of the next one.

14.2.4.3  Improving the use of essays
It is particularly important when setting essays to ensure that students know what they are being assessed on. Race (2001: 57–59) suggests a number of approaches, which are adapted below to the language learning context.
	The first is to involve students in looking at examples of past essays, and applying assessment criteria. This might be linked to peer assessment of each others’ essays (see Module 13, section 13.3.2 on peer assessment). 
Students need to know what exactly is required by an essay title. It is therefore a good idea to break up language essay questions into sub-questions, with the marks available for each section clearly indicated. This, along with a clear indication of target word length, will help to avoid verbosity and will provide some assurance and guidance to those who have difficulty coping with academic essays. This approach is especially advisable if the bulk of the marks are to be awarded to quality of language.
	Particularly if you are teaching students majoring in a non-Arts discipline, it is good preparation for the stresses and strains of a timed essay in exam conditions for students to practise writing essay plans, introductions or conclusions in the FL, either during class time or as homework. In the time it takes to write a full essay, students might be able to do five or six such tasks. Appropriate feedback on these should ensure that when they have to write a full essay, students are better able to concentrate on language and style and not worry about structure or the marshalling of arguments. 
	Linked to this, it can be helpful for all students to spend some time brainstorming how to respond to typical essay titles. Even in L1 this can be useful preparation; in the FL it can have the added advantage of revising/rehearsing relevant vocabulary. 
In general, it is not a good idea to overuse standard academic essays as this will advantage those who happen to be good at planning and writing them. So try varying your approach with other writing tasks, such as:

•	providing counter arguments to a paragraph of FL text that is 
arguing for a particular issue; 

•	developing an argument begun by a short FL text; 

•	writing an introduction of a specified length to an FL essay title.   

In giving feedback, be mindful of how what you write (or say) can help students to improve their essay writing. At the same time, try to reduce the time you spend giving different students similar feedback, for example by producing standard feedback sheets (see Module 13, Appendix 1 for an example, also Module 13, section 13.1.7 on ‘Reducing the marking burden’) and using an agreed system of error annotation (see 14.2.5 for examples).

14.2.4.4  Setting appropriate titles
Traditionally, university language essay exams have featured quite a wide choice of questions, in the belief that this was fairer to all students as it catered for their inevitably varied interests and knowledge. However, this can reduce severely the reliability of essays as a testing technique (see Module 13, section 13.1.4.2): the wider the choice of questions, the less reliable the assessment instrument becomes. 
	For this reason, rather than just giving an essay title such as ‘Write an essay on the role of the German President’, the following will provide more control over what is written, making the task a more reliable indicator of the student’s ability to write in the FL and improving the reliability of marking across the group being tested:

Discuss the role of the German President, paying particular attention to the following aspects:
    - how elected and period of office;
    - domestic responsibilities; 
    - external role;
    - relationship with the Chancellor and government;
    - legislative function.

Activity 12
Now consider the following essay titles and their appropriateness in an FL  exam:

1.	Imagine you are a French explorer. Write a diary describing 
your adventures at the South Pole.

2.	‘Pride comes before a fall.’ Discuss.

3.	‘The Germans are rightly proud of their education system.’ 
Do you agree?

4.	Discuss the pros and cons of the UK adopting the Euro.

5.	You are on a group visit to a Spanish university and are 
asked to address your hosts in Spanish or behalf of your 
group. Write out your speech.

Click on ‘Commentary’ for feedback.

This activity emphasizes the importance of ensuring that in setting language essays we are not testing things we do not mean to assess. The following title for an essay at intermediate level suggests how this might be done:

Compare the positive features of schools in England and Germany. Use all the points given below and express your own views in a conclusion. Write about 300 words.
England:
no school on Saturdays
mostly comprehensive
sixth-form colleges
variety of extra-curricular activities
Germany:
no uniform
breadth of subjects post-16
no public exams
free afternoons

It might be objected that such an approach tells students what to write; but, if the main aim of the essay is to test the ability to write in the FL, we should not be bringing in extraneous factors such as general knowledge. In formulating such questions, it is obviously important to use notes rather than whole sentences so as not to provide too much linguistic help with the task. However, a title such as this is far preferable to a bare ‘Compare and contrast the English and German school systems’, which places unequal demands on exam candidates, in particular. 
Moreover, an ‘unpacked’ title such as the one above, or the earlier one about the German Presidency, allows the tutor to exercise greater control over what is written. This might offend some on intellectual grounds, but it does make the task a more reliable indicator of students’ ability to write in the FL, and this is, after all, what we are supposed to be ascertaining when setting FL writing tasks.
	
14.2.5  Marks on scripts
(See also Module 9, section 9.4 for ideas on marking written work.) 
As noted in the Introduction to this module, testing at University involves not just formal exams for which students will usually not receive their papers back, but also less formal class tests and coursework which they will have returned. This section is concerned with the latter and what the tutor can do to help students learn from written tests and assessments. The comments do, of course, apply to the marking of any written work, whether assessed or not. 
A difficult decision that faces all tutors when marking written work is how to avoid overwhelming students with too many annotations. We must ensure they receive adequate feedback if they are to improve, but too much red pen can be demoralizing. A general guiding principle is that marking should always aim to promote learning. Module 13 (13.5.2) 
discusses different approaches to this, including the degree to which students can be expected to self-correct.
	A key instrument is the use of a list of standard annotations that you can use when marking students’ written work and to which students can refer when they receive their work back. Refraining from writing in explanations and corrections in every case will save you time and effort, but it will also encourage independence in your students since, especially if you do not provide a corrected version for them, they will have to work out the nature of their error and try to correct it themselves. Here are three such systems, one in German, one Spanish and the third in English (for students of French), which should be readily adaptable to other languages:

(a) German  

A	Ausdruck 	
Art	Artikel	
Gen	Genus	
I	Inhalt	
K	Kasus	
Kon	Konjunktion	
Konj	Konjunktiv	
MV	Modalverb	
P	Präposition	
R	Rechtschreibung	
S	Stil	
SB	Satzbau	
ST	Satzstellung	
T	Tempus	
W	Wortwahl	
Z	Zeichensetzung	

(b) Spanish

ac	acento	
adj	adjetivo	
adv	adverbio	
art	artículo	
con	concordancia	
conj	conjunción	
gen	género	
m v	modo verbal	
num	número	
pers	persona	
prep	preposición	
pron	pronombre	
sin	sinónimo	
t v	tiempo verbal	
v i	verbo irregular	
v r	verbo reflexivo	

(c) French

ADJ / ADV	Adjective used when adverb needed and vice versa	
ADJ / N		Adjective used when noun needed or vice versa	
AG 		Agreement: verb does not agree with subject or adjective with noun	
AGPP		Agreement of past participle	
ART		Article omitted when needed, or used when not needed	
AUX		Auxiliary: “être” used instead of “avoir”, or  vice versa	
BW / MD		Badly Worded / Mal Dit	
CON		Construction: verbal construction inadequate	
EE		English Expression: English expression translated literally	
EW		An English word is used instead of the French one	
G		Gender: feminine article with masculine word and vice versa	
HS		Hors Sujet: irrelevant material	
PREP		Preposition: the preposition used is incorrect	
PRO		Pronoun: the pronoun used is not appropriate	
REFLEX		Reflexive verb needed or used when not needed	
REG		Register: language inappropriate in the context	
REL		Relative pronoun (wrong one used)	
REP		Repetition: the same word or expression is used twice or more	
SP		Spelling: there is a mistake in the way the word is spelled	
T		Tense: the tense of the verb is not appropriate	
VOC	
	Vocabulary: the word or expression chosen does not express the right idea	
WM		Word Missing	
WO		Word Order: the words are in the wrong order	

(All examples from Centre for Modern Languages, University of Birmingham)

Ideally this will be a department- or language-wide system that all colleagues use, so that students do not have to cope with more than one system.
	In the same way, the department should have an agreed system for weighting various errors in marking students’ work. For example, in a punitive scheme, deduct:

½  for an orthographical or spelling error;

1 for an error with prepositional use or a case ending error;

1½   for a more serious error (eg misuse of tense).

Alternatively, in a positive scheme, with a criterion such as ‘half of the structures used are correct’, there needs to be clear agreement about what  counts as a structure and whether repeat errors are to be disregarded, etc.
	Once agreed, such schemes should be shared with students and, above all, consistently applied, not just over a single module taught by one tutor but across all the language modules taught in the department. This allows students to perceive fairness in tutors’ approach to marking and ensures in practice that students are more likely to be penalized or rewarded in the same way for the same mistake or the same correct use of language.

14.3  Assessing speaking
This section considers the most common approaches to assessing speaking skills in higher education. It does not seek to provide a comprehensive list of test types; for that, readers are referred to Underhill (1987: 44–87). The first four sub-sections deal with the most common forms of oral assessment, namely role play, presentations, oral interviews and student discussions. The final section then briefly reviews alternative approaches.
	
14.3.1 Role plays
Role play seems to be an exercise that students either love or hate, with older learners tending to be more resistant to it as a learning and assessment technique than younger ones. Extroverts are likely to take quite easily to it (though even here there are exceptions – see below), but not all students are extrovert or anything like it. It is sometimes said that to be good at role playing you need to be a good actor, moreover one who can act without relying on a script. This is a lot to expect of anyone; but to act out a role with an examiner who (a) has far more experience of language role-playing exercises, and (b) is much more proficient in the target language, is an extremely challenging thing to ask a student to do. 
	We need to bear all this in mind when considering the use of role plays. We must also carefully consider the role we create for students: is it something they can relate to from their own experience and sphere of existence? (eg don’t ask them to pretend to be an astronaut). And is it something that they will find reasonable or plausible? (eg don’t ask them to play the role of a world-famous actor or footballer, however much they enjoy the theatre or football themselves).

Activity 13

1.	Think of two role plays for students on the second year of a degree course in modern languages. They should be related to the topic of housing in the target country, a topic intended to prepare students’ for their forthcoming period of residence abroad. Make sure your scenarios are realistic and involve things one could expect students to have experience of. In one role play, students should be dealing with a complaint or enquiry, in the other they should be trying to obtain information or documentation from someone in a position of authority. 

2.	Decide how you would mark these role plays. What criteria would you employ? How could the criteria account for the different demands of the two role plays?

Click on ‘Commentary’ for feedback on this task. 

	The usual approach to role play is to give the student just before the test a set of instructions explaining what he/she has to do. For example:

You are on a visit to Paris. You want to visit the Palace at Versailles and so you go to a tourist information centre. Ask how to get there, find out the cost of travel and of entry to the Palace. Ask how long it will take to get there and see if the person serving you has any advice on when you should go. Describe what you are going to do and make the necessary booking.

Providing such instructions in L1 will avoid any ambiguity about what is required, and with near beginners this approach may be the only practical one. Otherwise, it is probably more common for instructions to be given in the FL. However, it should be noted that this can often provide linguistic support (eg, in the above scenario, ‘Palace’, ‘costs’, ‘advice’), which may or may not be welcome: with weaker students it might be desirable, while in other circumstances it could make the task too easy. One way round this is to phrase the instructions in more general terms: 

You are on a visit to Paris and want to go to Versailles. Find out from the person at the tourist information centre details of travel, cost and availability, and make the necessary booking.

Role plays are an effective means of involving students in particular types of language use, ranging from basic social situations to complex negotiating ones, such as convincing a company representative in an interview that you are the right person for the job. They are also the best way to test students’ ability to ask questions, to use certain social address formulae, to vary linguistic register and to employ functions such as persuading, objecting, explaining, etc. 
Apart from the type of low-level situation above, role play can be made more complex by including unexpected elements or ‘twists’. Here the examiner throws in something the candidate is not expecting, to see how he/she copes with the situation linguistically. For example, consider the following scenario:

You have lost the keys to your Spanish flat and you need help to get back in. Telephone the number of the town’s only locksmith and try to make arrangements for someone to come and change the lock. Insist that you need help immediately. You must try to negotiate a price for the work. The examiner will play the part of the locksmith.

The examiner in such a situation might have the following instructions:
•	be reluctant to help;
•	suggest the student breaks in;
•	say that you can only come the next day;
•	say it might be possible to come that evening but that the price would be double the normal rate, etc.
In this way candidates are forced to employ a variety of strategies and compromises, and to ‘think on their feet’.
	A variation on role play is to give students several short situations in which they have to supply an appropriate short response (usually just one sentence). These are essentially tests of functional language such as inviting, complaining or enquiring. For example:

1.	You want to ask an Italian friend to a party. What do you say?

2.	You want to buy a cheap camera from a shop in Madrid and ask the assistant for advice. What do you say?

3.	You are not happy about the service in a restaurant in Lyon. What do you say to the waiter?

These have the advantage of providing strictly controlled situations, thus enabling good coverage of a syllabus and ensuring validity, while allowing a fairly tight marking scheme with a limited range of acceptable answers, thus increasing the test’s objectivity and reliability (see Module 13, section 13.1.4.2). 

14.3.2  Presentations
A presentation in the FL is both a valuable formative learning tool and an effective form of assessment. Delivering a presentation to a group is a key transferable skill and students readily see its value and gain immense satisfaction from completing the task successfully. Knowing one is going to have to perform in public is also a great motivator and students will usually invest considerable time and effort in preparing and understanding material on the topic and in rehearsing their presentation, with huge benefits in terms of independent reading, deep approaches to learning and refining of oral performance (Klapper, 1991). In assessing speaking, it is vital to elicit a sufficient sample of candidates’ speech for assessment to be made; presentations provide this in abundance.

Activity 14

Think back to student presentations in the FL that you have experienced, either as a student or as a tutor. What do you remember about the experience? Were there any problematic aspects to them?

Presentations require considerable support to be given to students in the preparation phase, in particular guidance on techniques for employing visual material, voice projection and other paralinguistic features. Generally, students find them stressful affairs, and therefore, if they are to be employed for summative assessment, the ground needs to well prepared and students need a gradual introduction to the activity, with at least one ‘dry run’ before the real thing. Ideally, they should be given the opportunity to see themselves on video before the actual assessment, so they can get some sense of how they look and sound, and can work on the weaker aspects of their performance. Owing to the public nature of the task, oral presentations, far more than written work, expose the weakest students, and some presentations can be truly painful experiences (faltering delivery, incoherent content, poor pronunciation and intonation), from which others in the class  gain very little. In order to guarantee an acceptable minimum level of performance, it is therefore important to ensure weaker students have received every help and that they are encouraged to rehearse this task with particular thoroughness.
	If presentations are to be followed by ‘questions from the audience’ which the presenters are expected to answer in the FL, the task can help to develop broader oral skills. This can act as a useful supplementary means of assessing their knowledge and their ability to use unrehearsed language, something that is especially important if the presentation has revealed substantial rote learning of material. You might choose to reserve some marks in your mark scheme for how well students handle such questions.
Individual presentations are, of course, very demanding and most students prefer to contribute to group presentations. For the tutor, however, 
this is less satisfactory as it makes it more difficult to assess individual contributions and allows weaker candidates to ‘hide’ behind more able group members. In such circumstances it is probably fairest to award a single mark for the group performance which is added to an individual mark for each group member, to ensure a degree of differentiation. Making a video recording is the most effective way to assess such group presentations. 
	Practicalities (eg large numbers) will sometimes dictate that presentations cannot be performed in front of the whole class. Generally, however, it is beneficial to students to watch their peers perform as it allows them to see in what ways they can improve their own performance on such tasks.	
	It is even more important than with other tasks to ensure students know and understand the criteria to be used to assess their presentation. They need to know which aspects the examiner will focus on and what marks will be awarded to each category (see, for example, the oral criteria for presentations in Appendix 2). A good way to familiarize students with both the task and the criteria is to show them an example of a presentation on video from a previous cohort and to get them to mark the performance using the set criteria. To really involve students in the process, you could ask them to formulate or negotiate their own marking criteria; although this may not be practical if, as there should be, there are agreed departmental criteria for such tasks.
	It is generally acknowledged that when students are involved in assessing each other’s work, they tend to approach the task very seriously and conscientiously. You might thus consider an element of peer assessment for presentations – eg 50% students’ mark, 50% lecturer’s mark (see Module 13, section 13.3.2 on peer assessment). This should ensure students do not ‘switch off’ when others are presenting, as often happens. Having to mark each presentation will force them to attend closely to both the content and the language employed, and as a result they are likely to gain more from each presentation than they otherwise would.

14.3.3  Oral exams
Most people make little distinction between an oral conversation and an oral interview (cf Underhill, 1987: 45–46 and 54), and in universities, in practice, there is not usually a great difference since the two approaches tend to merge. Interviews, with their more closely defined list of specific questions and with less continuity, are perhaps more appropriate for lower- to intermediate-level learners. At higher levels, such tightly controlled approaches to question and answer do not allow students to display their oral ability to the full, therefore the examiner needs to withdraw a little from the exchange. As Underhill says (1987: 56): ‘Filling conversational pauses naturally and correcting one’s errors […] are two of the features that distinguish higher-level from intermediate-level learners. Ultimately, the interviewer must be prepared to yield the initiative to the learner.’	

Activity 15

Before reading any further, take a moment to think about your questioning technique in oral exams:
1.	Do you prepare your questions in advance? 
2.	Do you consciously vary the type of question you ask? 
3.	How can one ensure reliability (cf Module 13, section 13.1.4.2) in oral exams?

While no one would want to put a straitjacket on examiners’ questions, it is only fair to ensure that students are asked a comparable number and range of questions and that the questions are of a broadly comparable level of difficulty. It is all too easy, especially if you are on candidate number 20 and are getting bored or tired, to allow a conversation to drift into (narrow) areas you may not have covered with other candidates, or to allow questions to follow your own areas of interest. Of course, if a challenging topic is allowing a student to demonstrate his/her linguistic ability to the full, it is only right to allow such a student to show what he or she can do, but the topic should not dominate the whole oral and you should bring the conversation back to the broadly common agenda.
	If you are doing an oral for the first time, it is sensible to prepare a list of sample questions in advance: it is surprisingly difficult to focus on accurate assessment, while also keeping track of the conversation and checking you have covered all the relevant areas of a topic. In any case, all oral examiners will find it helpful to have in front of them a checklist to ensure they do not omit any key use of language they are aiming to assess. For example, the following is a list I have used with an intermediate IWLP group:

Can relate what he/she is studying.
Can explain choice of degree programme.
Can narrate a sequence of daily events.
Can describe route to university.
Can describe place of residence.
Can narrate an event in the past.
Can talk about plans for the summer.

This is not as prescriptive as a list of set questions and allows flexibility to cater for differences in individual candidates, but it does mean each student is prompted to produce the same sort of language and it thus ensures you are testing the same things and increasing the reliability of your marking.
	Given the difficulty we all have in focusing on several tasks at once, it is advisable to have two examiners involved, preferably one native- and one non-native speaker. Although this may be a little more daunting for the student and makes it even more important that the interlocutor appears friendly and supportive to the student, it means he/she is sitting opposite at least one familiar face (ideally his/her tutor) and is therefore likely to be a little more relaxed than with a stranger. The second person can either be the assessor who does not get involved in the exchange, thus allowing the colleague to focus solely on asking questions and keeping the conversation going, or else both can take it in turns to ask questions and both can be involved in reaching a mark. While it is clearly more staff-intensive, this approach increases reliability in marking since, at any one time, one of the examiners can give undivided attention to the student’s performance.
	If staffing levels do not permit the above approach, it is important to record all orals, either on video or audio tape. This means you have something tangible to support your decisions, in case a student queries a mark, for example. It also provides evidence for external examiners or for QA purposes. Furthermore, it allows you to review any interviews in case of uncertainty over, say, borderline marks, where memory may prove unreliable. Tapes can be especially important with large numbers of oral exams, when fatigue can set in and consistency across the cohort may be less certain; a quick review of earlier tapes can provide reassurance or act as a corrective.
	Question technique is obviously crucial in an oral. You should avoid so-called ‘closed’ questions requiring simple yes/no or equally brief answers, and employ as wide a range of open questions as possible. These can be any one of the following types:

simple or literal;
language practice;
interpretative; 
analytical; 
inference;
personal response; 
evaluative. 

(Module 8, section 8.4.6, illustrates the use of these different types of question.)
	Generally speaking, who?/when?/what?/where? questions are more straightforward to answer than ‘how?’ questions. The aim is to get the students to talk, to show what they are capable of, and your role is to ask appropriately framed questions that prompt a full response, provoke thinking and, hopefully, set up an intelligent exchange of views. You must therefore keep yourself under control: the oral is not about you talking or demonstrating your knowledge, nor is it about nit-picking. As a rule of thumb, with anything other than the weakest candidates, if you are talking as much as the student, you are not doing your job properly.
	It is also important to elicit questions from the students themselves – asking questions is a key skill in learning a language and surviving abroad. As an examiner, you might simply include a section where you say: ‘Now I’d like you to ask me a few questions (eg about my job, my holidays, my views on the EU, etc)’.

14.3.4  Student-student discussion

Activity 16

In what way do you think the role of the examiner in an oral group task differs from his/her role in a one-to-one oral exam?

A useful variation on the traditional oral exam is the oral group task, a discussion between students in which the examiner is not involved. This is a good all-round assessment of the ability to communicate in a foreign language in a realistic setting. It is the responsibility of the students themselves to develop the discussion and to reach agreement on some outcome. Ideally, the task will have no single ‘right’ answer and the topic will involve various possibilities. To avoid the situation where students have to depend on general knowledge or their own inventiveness, they might be asked in advance to look through relevant source material to help inform the discussion. Such stimulus material needs to be succinct and straightforward to ensure it does not become a test of reading, and it should offer sufficient options which must all be equally feasible. Underhill (1987: 49) suggests some likely tasks:

Choosing a suitable educational course for a person with specified interests from a range of possible courses.

Choosing a suitable holiday for a particular person or family from brochures.

Choosing the best bid for a contract from a number of bids.

Choosing the best candidate for a job from several applicants.

Deciding which applications for bank loans should be granted, when the total number of loans requested exceeds the amount of money available.

Discussing a proposal for a company (real or imaginary) to launch a new product or enter a new field of operation.

Although the examiner here can sit back and focus on assessment, without being worried about keeping the exchange going, it can be very challenging to assess up to five people simultaneously. Partly because of this and of the need to ensure all candidates have the same opportunity to contribute, the exam needs to be longer than the traditional oral but it should still prove more economical than conducting one-to-one interviews.
To avoid the danger of students not contributing enough to the discussion, a structured approach might be adopted, as for example in The Open University’s speaking test, where each member of a group of four has to make a brief initial ‘pitch’, in which he or she outlines a view or position on the topic, prior to a general discussion to which all group members are expected to contribute. Even with this approach, however, some performances may still prove easier to assess than others, so it is a good idea to award a mark first to the candidates who contribute most, before devoting most of the marking time to the individual contributions of the least talkative. 
It is particularly important in such group tasks that in advance of the test students are made fully aware of the criteria against which they are to be assessed, ie that they know it is not just their presentation of facts or information that count but how they justify their position in relation to that of others. 

14.3.5 Other oral test types

Activity 17

Before tackling this section, write down all the oral tasks you can think of and allocate them to one of the following three headings:

Tasks I already use in my teaching

Tasks that I could use in my department

Tasks that would not be acceptable in my department

In this section we briefly consider alternative approaches to assessing students’ spoken output. Not all of them may prove appropriate or practicable in every university setting, but they all have distinctive features that might  serve to complement the more traditional approaches outlined above.

14.3.5.1  Integrated skill tests
In terms of authenticity and closeness to real-world language use, tests which involve more than one skill are increasingly thought to be preferable. Such tests are, however, more demanding on the setter, and despite their high validity (see Module 13, section 13.1.4.1), they can lead to confused measurement, especially if performance on one element (eg a prior reading or listening task) interferes with performance on a subsequent speaking or writing task. The creation of appropriate criteria can also be more challenging.
	One such integrated task for oral testing involves candidates in renarrating the content of something they have heard on tape. The mark scheme for this can be quite specific, listing the points the candidate has to reproduce, but it needs to be supplemented by criteria for the quality and range of language employed. An alternative is to give candidates a text to read for a few minutes, to take the text back and to ask them to renarrate or summarize the text in their own words. The integration of reading and listening skills involved here can be furthered if candidates are allowed to retain the text and the oral examiner then asks specific questions on aspects of the text – eg ‘What does the phrase ‘x’ in the last sentence mean?, or: ‘Can you explain in your own words what is meant by the second sentence?’, etc. 

14.3.5.2  Picture stimuli
A useful way to vary the traditional approach to oral exams is to start with a picture which the student has to describe. After he/she has spoken as freely as possible, the examiner might add specific questions to prompt further responses, in particular to elicit interpretation of the situation. These latter questions can lead into more challenging areas and might be used to stretch the more able (see 14.3.3 on question types). A variation on this is to provide a story in pictures (eg a cartoon) that the student has to narrate following a few minutes preparation.
	Both these approaches are an effective lead-in to or warm-up for freer conversation and avoid ‘contamination’ by any other language skill, ie they do not require candidates to read or listen as well. If they are intended to serve just as warm-up activities, then it is sensible not to allocate many marks to them. Alternatively, they could form a principal means of assessing oral ability and indeed might be an effective way to combine a degree of validity and reliability: all candidates have to talk about the same topic or event and use broadly similar language (especially vocabulary and tenses). To avoid the task foundering on unknown items of vocabulary, difficult but essential words might be provided beneath the pictures. Finally, it is obviously important that all pictures used are clear and unambiguous. 

14.3.5.3  Information-gap activities
In these exercises, pairs of candidates are given incomplete information on a card and one student has to elicit orally from the other the remaining information to complete the task. Candidates can ask each other questions, ask for clarification and ask their partner to explain things differently if they do not understand, ie employ the interaction and improvisation typical of natural conversation. To work well, the task requires students of broadly comparable proficiency in the FL, since if one is appreciably better than the other, he/she may dominate the exchange and not allow the other sufficient opportunities to contribute. A variation on this is for the teacher-examiner to take the other part; although this may be more demanding for the candidate, it at least ensures all candidates deal with a partner who reacts in a similar manner, thus improving the reliability of the test.

14.3.5.4  Explaining/providing instructions
A simple but deceptively challenging oral task is for candidates, after a few minutes preparation, to describe in detail how something works. This could involve a well-known everyday procedure or object. Underhill (1987: 69) lists some possibilities:

  -  How do you make a good cup of tea?
  -  Describe a bicycle.
  -  Explain how to send an e-mail.
  -  Give instructions for using a public pay phone. 
  -  Explain how you would advise someone to look for a job.

One might even allow students to choose one such topic from a list so that no one is disadvantaged (eg they might know nothing about bikes). Either way, the task provides the opportunity for free expression and choice of language but allows a degree of objectivity in the marking (ie certain steps need to be covered in each case.) For the task to work properly, students should be forewarned that their response must include a specified number of steps.
	At higher levels, the same basic approach might be adopted by setting candidates a more challenging topic of general interest (eg the Euro, private education, the health service) and having them speak on it after a brief period of preparation. The short presentation can be interspersed with and extended by questions from the examiner to follow up specific points, clarify certain issues or challenge the candidate on his/her views. As above, candidates might be given a number of titles to choose from, but care needs to be exercised here, since this can result in things other than language being tested, eg general knowledge, logical processes, imagination or creativity (cf section 14.2.4.4 on setting essay titles) and a range of titles can result in students adopting a range of approaches, thus making it difficult to achieve reliability in marking. 

14.3.5.5  Reading texts out loud
Here candidates are given a text or texts to read aloud, following a few minutes preparation. This is a controversial task. Research suggests it is not conducive to effective language learning (see Module 8, section 8.4.1). With regard to testing, it is very dependent on a candidate’s reading ability and, since it can fairly easily be developed through coaching, it would not seem to have much to do with someone’s speaking ability. However, it is an easy task to set up and mark, and is very effective in assessing students’ pronunciation, intonation and their ability to recognize appropriate stress within and across sentences. In a language with variable stress patterns, such as Russian, it can also serve as a good test of knowledge of stress at word level. Other than for these purposes, it is not to be recommended as an oral assessment instrument as it is susceptible to interference between the skills of reading and speaking. Furthermore, even between native speakers there would be major differences in performance on a task such as this, suggesting it is not the ideal task for FL learners. 

14.4  Assessing listening and reading
In this section we look at the testing of the two so-called ‘receptive’ language skills. There are a number of principles common to the testing of listening and reading, however there are also several distinctive issues in each case; for this reason they are treated separately in the first two sub-sections. There follows discussion of the use of open-ended questions and summaries as  techniques for testing comprehension. We then explore the at times controversial issue of target-language testing. And, finally, we consider the use of multiple-choice questions in listening and reading tests.

14.4.1  Listening 

Activity 18

1.	Take a few minutes to consider the place of listening in the department where you work. Is it formally assessed in exams? Does it feature in assessed coursework?

2.	Think back to your own language learning. Was listening assessed as a separate skill? If so, how?

Despite the widespread acceptance of a skills-based approach to teaching and learning languages, it is surprising how often students’ listening ability is not specifically assessed, appearing at best as an implicit skill in the oral exam. This is a pity, as many students come to university now with reasonably well developed listening skills, having practised them intensively at GCSE, ‘AS’ and ‘A2’ levels; for students struggling with the grammar of a language, listening assessments offer an opportunity to perform creditably in at least one area of their work.
	Before we consider the different types of listening task that can be used in tests, here are some general principles to observe when you are preparing to assess listening:

•	Make sure that tasks match the teaching and learning that has taken place: listening to a discursive monologue when the module has only featured dialogues or interviews, might be deemed unfair and poor pedagogy. 
•	Take care not to overload memory; consider carefully how often the stimulus material should be spoken or played to candidates, and how frequently pauses, if any, should be included. 
•	Distinguish between listening for gist and for detailed understanding of specifics; criteria ought to reflect this distinction.	
•	In preparing test items, work from the tape (if there is one), rather than a transcript. This will avoid unreasonable demands being made of the students; it will also help with the next point.
•	Keep items to be tested a reasonable distance apart in the recorded or spoken test. To do otherwise is unfair and can lead to consequential errors: if students miss an item, they can still be waiting for information that has already gone.
•	Allow students to look through questions first before they listen to the material.
•	If students have to answer in the FL, don’t include writing ability in your mark scheme; including productive skills will make the measurement of listening ability much less accurate and reliable.

14.4.1.1  Task types
Exercises and tasks for listening are just as applicable to the testing of reading, so this section will cover both skills. Tasks can basically be divided into five main groups:
•	graphic representation (GR);
•	open-ended questions (OE);
•	multiple-choice exercises (MC);
•	true/false statements (TF);
•	ordering/matching questions (OM).

We shall briefly consider each of these in turn.
1.	Graphic representation is especially common in the early stages of FL learning. It denotes the drawing or recording in some other graphic format of information contained in spoken and written texts. It has the advantage of not requiring students to write in the FL but can sometimes be problematic to mark if a student’s drawing is not clear or is not quite accurate.
2.	Open-ended questions are simple questions on a text, either in L1 (first language) or the FL, which students answer freely in their own words or by quoting material in the text. Questions may be placed before or after the text is heard or read. Questions are quite easy to formulate but students can simply quote bits from the text in reply, and variable responses (eg very long or very short, very accurate or very inaccurate use of the FL) can make them difficult to mark.
3.	Multiple-choice exercises offer students a range of possible answers, thus removing the need for students to write anything substantial. Although easy to mark, these can be difficult to write (see section 14.4.5). Moreover, they are not always ideal for listening tests as they can require students to retain in their heads the various alternatives, while at the same time listening to the text.
4.	True/false statements are easier to devise than multiple-choice exercises  and they are very easy to mark. They do, however, obviously make it easier for students to the guess answer.
5.	In ordering/matching questions, students have to put items in the correct order, link appropriate text elements or ideas with each other, or fill in tables with numbers, letters or symbols relating to the text. They have the advantage of requiring students to listen to or read the text very carefully. 

Numerous specific tasks and exercises fall under these five basic headings and the following section and section 14.4.2.1 consider a range of them.

14.4.1.2  Listening tasks
The choice of possible comprehension exercises for listening will depend on:
•	the nature of the listening situation (public or private, listening for particular or general information);
•	the style of listening (ie how the students have to listen), either selective, detailed or global;
•	the type of understanding the tutor wishes to assess (general, specific, limited, partial or complete). 

Consider the following table:

Listening situation	Listening style	Target understanding	
Interested in particular information (eg in weather forecast, whether it is going to rain where you are)	

Selective	

Limited and partial	
Want to understand all aspects of a TV programme on a topic you are currently studying	
Detailed	
Specific and complete	
General interest in topic, want to grasp main ideas only in a radio report	
Global	
General	

(Based on an idea in Bolton, 1996:158)

Activity 19
Now use the abbreviations introduced in 14.4.1.1 (GR, OE, MC, TF and OM) to indicate in the following table which exercise type each of the tasks described belongs to:

Test format
	Student task	Exercise type	
Spoken text and clock faces	Enter times heard onto clockface		
Picture and oral statements about it	Decide whether each statement is correct 		
Spoken text and map with key to symbols	Put symbols on map in relation to what you hear		
Several pictures and one oral statement	Tick the appropriate picture		
Spoken text and written L1 statements	Decide whether each statement is correct		
Spoken questions and several written answers on each	Tick the appropriate answers		
Spoken text and written key words relating to it	Tick which of the ideas occurs in the text		
Spoken text and L1 questionnaire	Write in L1 words on questionnaire		
Spoken text and street map	Transfer data to street map		
Spoken text and several oral statements	Tick the appropriate statement		
Pictures and oral statements	Decide which pictures go with which statements		
Spoken text	Draw an object described in the text		
Spoken text and oral statements about it	Decide whether each statement is correct		
Spoken text and jumbled pictures	Put pictures in correct sequence		
Spoken text and several written L1 statements 	Tick the appropriate statement		
Spoken numbers or dates	Write down figures		

(Based on typology in Doyé, 1988, and task in Bolton, 1996: 59–61)

Click on ‘Commentary’ for the completed table.

Space does not allow us to consider each of these activities in detail, but readers are referred to Hughes (1989: 134–40), and Weir (1990: 43–51) for further discussion of approaches to listening assessment and test types.

14.4.2  Reading 
We begin with some general points to be considered when producing tests of reading ability:
•	Avoid texts and questions in which general knowledge can be used to get the correct answer.
•	Include as many different passages as possible: this will increase the reliability of the test, as students will get a number of ‘fresh starts’.
•	Include some items that focus on micro-skills, such as vocabulary or idiom. For example: ‘Find a single word/a phrase/a sentence in the first part of the text that means the same as…..’
•	If you are testing scanning (see Module 7, section 7.2.2), you need texts with a good deal of separate or discrete information, rather than a discursive text.
•	As a rule, you should present questions in the order in which the answers can be found in the passage; but see the next point.
•	At intermediate levels and above, don’t just include scanning questions but  also ones that require a detailed appreciation of argument or an ability to link different parts of a passage to each other.
•	If using the FL in your questions, ensure you do not make identification of the answer too easy.
•	As with listening comprehension, if the FL is to be used in answers, don’t include writing ability in your mark scheme.

14.4.2.1  Reading tasks
As with listening, the choice of exercise for reading will depend on the nature of the text and the type of reading and understanding the tutor wishes to assess. To clarify this, see if you can fill in the types of exercise, ie graphic representation, open-ended, multiple-choice, true/false, ordering/matching (cf 14.4.1.1), and the types of reading (ie gist, detailed, global, skimming and scanning)  usually associated with each of the three text types in the table below.

Activity 20

Text type	Exercise type	Type of reading	

Multiple points of view (eg questionnaires, interviews)	

 _______________	
Gist and detailed reading	

Lengthy, self-contained texts (eg stories, reports, factual articles)	
(1) Open-ended

(2)  ____________

(3) True/false	
Gist, detailed, skimming and scanning

Detailed

____________	

Texts read quickly for specific information (eg timetables, TV guides)
	
(1)	_____________

(2) Open-ended
	
Skimming, scanning

Skimming, scanning	

(Adapted from Bolton, 1996: 38)

For the completed table, click on ‘Commentary’. 

Activity 21
Using the abbreviations introduced in 14.4.1.1 (GR, OE, MC, TF and OM) once more, indicate in the following table which exercise type each of the tasks described belongs to:

Test format	Student task	Exercise type	
Picture with written statements	Tick the correct statement(s)		
Written texts and beginnings/endings of sentences	Decide which sentence parts go together in relation to the text		
Questions on a text with, in each case,  alternative answers	Tick the correct alternative		
Several pictures and one written statement	Tick the appropriate picture		
Descriptions of people and key words about them	Write down the names of the people associated with the key words		
Written text with written statements	Tick the correct statement(s)		
Written text with written statements	Tick the appropriate statement		
Written text and accompanying table	Tick the facts mentioned in the text		
Written text	Underline sections of text important to a specific question (eg ‘arguments for’)		
Written text	Draw a plan of something described in the text		
Written texts and pictures	Link pictures to the relevant texts		
One picture and several written statements	Tick one of the statements		
Written texts and headings	Link headings to the appropriate texts		
Questions and answers	Match the questions with the answers		
Written text with map and key to symbols	Put symbols on map in relation to content of text		
Written text	Summarize main contents in L1		
Written text and jumbled sentences about it	Put sentences in correct order		
Introductory sentence or clause with several possible continuations	Tick the appropriate continuation		
Written text and L1 questions on text	Answer questions on text content		
Jumbled phrases/clauses	Put phrases/clauses in correct order		

(Based on typology in Doyé, 1988, and task in Bolton, 1996: 42–43)

Click on ‘Commentary’ for the completed table.

See Hughes (1989: 116–33), and Weir (1990: 51–58) for further discussion of approaches to reading assessment and test types.

14.4.3  Further ways to assess comprehension 
Section 14.4.2.1 on reading assessment contains reference to open-ended  comprehension questions and to summaries. These are by far the most common approaches to reading assessment in university language departments and they therefore deserve a few additional comments.

14.4.3.1  Short answer questions
The advantages of this approach are that it usually ensures brief student responses and thus means the tester can cover more of the text(s) in the test. Unlike when answering multiple-choice questions (see section 14.4.5), the student has to seek the answer himself/herself so that it is more likely the correct answer will be arrived at for reasons of comprehension than any other reason (eg guessing in multiple-choice exercises). The approach allows the skills of inferencing, sequencing and comparing disparate parts of the text to feature in the testing process. The only real drawback is that unless care is taken in formulating questions, marking can be made more difficult and even unreliable by the presence of more than one acceptable answer, especially where students have to infer the answer.
Short questions can be answered in either students’ L1 or the FL. It is best not to mix the language of question and answer, so as to avoid translation impinging on the task and confusing the purpose of assessment (see also 14.4.4). Clearly, FL answers to FL questions are more challenging for students if, as should be the case, they are required to avoid simply repeating verbatim the original text. Such questions might thus be thought better for more advanced levels of proficiency, while L1 answers to L1 questions would be more appropriate to lower-level assessment.
Answers in the FL also raise issues for the tutor marking the work: is it the factual content alone that is being marked and is therefore the quality of the language to be ignored where it does not prevent communication of the message? Or are some marks to be awarded for the quality of language used as well as for content? If the latter, it is only fair to let students know exactly how their answers will be marked, and to make clear to them if, and how, they will be penalized for simply reproducing stimulus material, ie copying chunks from the text. On balance, introducing a quality of language mark threatens to confuse the purpose of the assessment, that is to measure comprehension, and it is better not to do it. 
Occasionally, under the misnomer of comprehension, one comes across the curious practice of students being asked to supply FL answers to questions on an L1 text. This requires a lot of groundwork on FL vocabulary and structures and is a very challenging task. In effect, it is a variation on prose translation, albeit one that allows students greater freedom in the language they choose to express their meaning; clearly, it has nothing to do with FL comprehension.

14.4.3.2  Written summaries
This is one of the most valuable FL learning tasks, one that can also serve as an effective assessment instrument, provided the aims are made clear and the criteria are precise enough. As a learning tool it is useful in that it involves students in comprehension, elements of translation and free writing. Sometimes referred to as précis, it is a key transferable skill that students are likely to require in their subsequent working life, and as such it can help to motivate students.
	Students might be asked to summarize:
•	an FL text in English;
•	an FL text in the FL.
These require different abilities and, in the learning process, different support. In terms of formulating assessment guidelines and criteria:

1.	FL–English summary is the more straightforward. Students have to distinguish main ideas from minor detail and organize them into a coherent text. Mark schemes should therefore allocate an impression mark to the quality of the summary. Although such an approach may slightly reduce the objectivity of marking, this need not be significant if the quality mark is limited to, say, 20% of the available marks. 

2.	As noted in the previous section, an FL–FL summary throws up problems with regard to how much of the source material it is legitimate to recycle. It  requires clarity about what value is to be placed on students’ rendering of factual content, on the one hand, and the quality of their use of the FL, on the other.

14.4.3.3  Free recall tests
In these exercises, students read a text; the text is then taken from them and they have to write down everything they can recall about it. It is important that students use L1 when writing, in order to avoid confusing the object of assessment. The approach has the advantage of omitting questions from the test and thus removing one variable that might distort assessment. The task is best marked by building a mark scheme based around distinct ideas contained in the text.

14.4.4  Target-language testing

Activity 22

1.	What is your department’s policy on testing in the FL?
2.	With comprehension questions, do you think there are grounds for insisting on exclusive use of the FL in both question and answer?

Assessment of FL listening and reading is closely tied to the question of whether L1 or the FL should be used as the medium of assessment. The key issue here is validity: we must be sure that what the test aims to assess is indeed being assessed, rather than something else altogether. For example, FL instructions can introduce a reading element into assessment.
	Page (1993) points out three problems relating to validity:
•	The difficulty of avoiding comprehension problems with instructions can limit the range of tasks to simple and unimaginative ones.
•	In setting reading assessments, the range of tasks can be severely restricted by the need to avoid questions in which the answer is ‘signalled’ or even given by the question, thus reducing the need for the student to show understanding of the text or what they are writing. This is the ‘Jaberwocky’ problem:

Q: What did the slithy toves do?
A: They gyred and gimbled.
Q: Where did they do it?
A: In the wabe.’ (Page, 1993: 6)

This can result in questions that focus on minor aspects of a text, rather than the key issues.
•	When students are required to answer in the FL, taking account of the quality of their language use threatens to invalidate the test, as language production becomes confused with the real issue, comprehension.
The argument often advanced in favour of FL testing is that it is done effectively in the EFL world. The counter argument is that the EFL world usually has no choice, whereas in FL teaching, with a common mother tongue, we are able to choose which language is more appropriate to a particular assessment situation.
	Despite such serious reservations, the past ten years have seen the spread of target language testing both to public exams at secondary level and university assessment. A major reason has been the danger of negative washback on classroom practice. Two reports from the School Curriculum and Assessment Authority (Neather et al, 1995; and Powell et al, 1996) have made the case for target language testing. The following list of possible approaches draws in part on these documents; some will be familiar from the taxonomies included in Activities 19 and 21 above. 

At lower levels:

•	choosing/ticking pictures and other visuals to show knowledge of phrases and single lexical items;
•	answering one-word multiple-choice questions;
•	ticking boxes containing single words;
•	answering questions that require a number as an answer;
•	writing single words or short phrases to complete a form, grid, chart or list;
•	completing gapped FL sentences – eg: ‘The reason for the _______ of the plan was lack of resources’;
•	correcting a list, phrase or sentence;
•	putting true/false next to a list of statements.

At higher levels:

•	completing sentences – eg: ‘The argument used to counter the proposal was that ___________________’;
•	finding synonyms or equivalent words/phrases in the text;
•	using context to provide definitions of words/phrases from the text;
•	drawing a diagram or map based on instructions in a text (eg ‘Complete the plan of the supermarket based on the information contained in the text’);
•	questions that require deduction rather than a mere reworking of the text;
•	carefully formulated mark schemes that stipulate the degree to which communication, as opposed to accuracy, is taken into account.

In listening, the demands of the activity mean that questions requiring FL writing beyond single words or short phrases, are likely to be too difficult at anything other than advanced levels. In all cases, the key is to provide instructions that are simple, to the point and that follow a predictable pattern – and preferably are standard from year to year.
In conclusion, there are clearly arguments in favour of both target language and mother tongue testing. Alderson (2000: 147) sums up the debate by saying that the most important thing is to make sure instructions are as comprehensible as possible, which usually (my emphasis) means using L1. However, it is clear that there are numerous ways in which L1 use in tests can be minimized. In the interests of continuity between teaching and assessment, and of ensuring positive washback on classroom practice, these techniques ought to be maximized to promote majority target language use.
But the interests of validity demand that dogma has no place here: where L1 will avoid ambiguity or a confusion of skills, it should be used.

14.4.5  Multiple-choice questions

Activity 23

Think back to any experience you have had of multiple-choice questions, either as tutor or student. Can you think of any disadvantages of using them in your current language teaching?

In section 14.1.3, we contrasted discrete-point testing with integrative testing. Probably the best known discrete-point test is the multiple-choice test. Over the years, this has been a popular method of testing reading and, to a lesser extent, listening. It does have distinct advantages over other test types, but it also has a number of drawbacks which you ought to be aware of before deciding whether to use it.

1.	Advantages:

•	Multiple-choice tests can be easily administered and can be done quickly by students.
•	They can be easily marked; with large numbers, they can be machine-read.
•	Since they usually include several possible answers, they reduce the opportunity for guesswork of simple true/false questions.
•	Mark schemes are very straightforward. They convey to students the impression of an entirely fair approach to marking, one more open to public scrutiny than that typically adopted for FL essays.

2.	Disadvantages:

•	It is much more difficult than you might think to write clear and unambiguous test items (cf 14.1.2). 
•	They are generally unpopular with learners as their purpose seems to be to catch people out, to entice them into choosing the wrong option (tellingly, the wrong options are known as ‘distractors’).
•	It is thought they might not always be a good measure of understanding, as the distractors can induce learners to choose an option they might otherwise not have thought of.
•	They restrict what one can test: it is especially difficult to come up with credible distractors for certain uses of language (eg tenses).
•	They only test recognition knowledge: being able, for example, to identify grammar items in a multiple-choice test does not mean the student can use grammar in context.
•	To be successful, tests need to be trialled so that those which fail to discriminate between candidates can be removed or amended. (For a detailed discussion of trialling procedures, see Bailey, 1998: 132–40.)

Owing to the laborious nature of such trialling, many think multiple-choice tests are only viable if they can be used as ’closed exams’ and therefore re-used in subsequent years. This probably excludes their use in terminal or end-of-year exams but would mean they could be used as progress tests in mid-semester. However, this would have the negative washback effect of not allowing them to be used for feedback purposes in class. 
	A significant area in which multiple-choice questions are widely used is computer aided assessment (Atkinson, 2002). Typically, in both listening and reading, students are asked to choose an option from a menu and, if it is not a test situation, they can receive instant feedback. There are clear limitations here as well, however, since extended reading tasks are difficult to assess on a computer owing to the limitations of processing long texts on screen.

14.5  Other forms of testing 
This section moves beyond the four skills to look at other testing procedures, some well established, others of more recent provenance. Cloze and C-tests have been in use for a number of years and the first sub-section considers  their purpose and efficacy. Grammar tests and translation tasks are widely used in university language programmes and approaches to these are considered in subsequent sections. Finally, we turn our attention to the use of projects, portfolios and reviews as increasingly popular alternative methods of assessing students’ language.

14.5.1  Using cloze and C-tests
The 1970s saw much discussion of the so-called ‘Unitary Competence Hypothesis’ (Oller, 1979), in essence the belief that language ability could not be broken down into component parts. According to this theory, it was not possible to measure reliably and separately a learner’s writing, reading or grammatical ability. Any such test would simply measure a unitary language ability. However, it was not long before the theory was shown to be false; as most language learners are aware, there frequently are differences between, for example, one’s ability to read and speak in a FL. 
All the same, these differences are usually quite small, since most language training ensures that skills develop at a broadly uniform rate, and there would seem to be some learning transfer between skills. Consequently, while rejecting the Unitary Competence Hypothesis, many test developers still felt that owing to the relatively minor differences between language skills, it was in fact possible to devise measures of overall language ability. Cloze and C-tests fall into this category.
They are two examples of an approach to language assessment that makes use of our capacity for predicting how an utterance will develop, for filling in missing information by using limited linguistic stimuli. (The most common such technique is gap-filling, employed very often, of course, in the practice and reinforcement of FL grammar.) Cloze and C-tests rely on a combination of ‘syntagmatic’ and ‘paradigmatic’ competence (cf Bailey, 1998: 59f). The syntagmatic competence refers to our ability to work out what part of speech is likely to be missing; for example, in the sentence ‘The policeman stepped out into the road and _______ the traffic’, we can assert pretty confidently that a verb is missing. Paradigmatic competence allows us to work out elements of the missing word’s meaning, ie the sorts of action a policeman might perform once he has stepped into the middle of a road.
The great advantage of this approach is that it allows the tutor to focus the testing on a single element (eg knowledge of appropriate verbs or nouns, or else some grammatical feature such as inflected adjectives or the use of articles). Moreover, marking is straightforward as there is usually just one correct answer, although with non-grammatical and thus open-ended items, it is essential to try the items out with colleagues or, ideally, students, to identify possible ambiguities and to help compile an appropriate mark scheme.

14.5.1.1  Cloze tests
Cloze tests are texts of a few sentences in length from which words have been omitted at regular intervals (eg every 5th, 8th or 10th word). Originally used to assess first language reading, they are also considered to be an effective way to assess overall proficiency in the written language of FL learners. 
Typically a text will begin with a complete sentence before the blanks start to appear in the second sentence. For example:

Activity 24
Fill in the gaps in the following text. Are all the answers obvious? Are there any you might have problems drawing up a mark scheme for?

Many able students are not very proficient at L1>FL translation. They discover that, although they have _____ understanding of the target language text, _____ simply cannot find the appropriate English _____ which to express the foreign language. _____ reasons for this can vary greatly, _____ it may simply be due to _____ knowledge of L1. It seems to _____  particularly true of able learners who _____ learnt the FL naturalistically, with minimal _____ to their mother tongue. In such _____, the FL is processed in ways _____ akin to the processing of the mother tongue.

A full version with feedback can be found in the ‘Commentary’.

	Marking of cloze tests can be based on the principle of there only being one acceptable answer, ie the original deleted word, or else any alternative might be accepted that is grammatically correct, fits the context and does not render the text incoherent. The first approach is the quicker and more reliable method: the marker has a simple right or wrong decision to make. The second slows marking down considerably but rewards candidates’ creativity, and if more than one marker is involved, it may detract from the test’s reliability since one person may deem something acceptable that another would not.
	A critical issue in writing cloze tests is to get the level of the text right: it must not be too difficult, as blanked words will make it even more inaccessible. Factors to consider include:

•	length of text;
•	frequency of blanks (the closer together they are, the harder they are);
•	familiarity with the topic, the vocabulary and the structures used;
•	sentence length;
•	time allowed.
(Bailey, 1998: 62)

The blanks in the text in Activity 24 were chosen on the basis of ‘fixed ratio deletion’, ie every 7th word. The alternative approach is ‘rational deletion’ in which something specific, such as past-tense verbs or conjunctions, is deleted so as to test students on just that item. Generally speaking, fixed ratio deletion makes for a harder test.
	Reservations concerning cloze tests include doubts about their reliability and validity. Fixed ratio deletion frequently results in isolated items that are impossible for the candidate to predict. Furthermore, it is not a consistent procedure. As Weir states: ‘If one changes the text, changes the deletion rate, starts at a different place or alters the scoring procedure, one gets a different test in terms of reliability and validity coefficients and overall test difficulty’ (1990: 47). This points to the need for rational deletion, where deleted items are consciously selected and maybe trialled to assess their difficulty. However, even this does not address the major criticism of cloze tests, namely that they are based solely on individual words, and that gaps can be filled by reference to the immediate context; consequently, reading skills above the level of the word, phrase or sentence are usually not assessed. 

14.5.1.2  C-tests
These variants on cloze tests were developed in response to precisely the above concerns, namely that in cloze tests students do not really have to process text above the level of the clause. A C-test involves blanking out the second half of every second word, starting from the second sentence of a short text (see Klein-Braley and Raatz, 1984; and Grotjahn, 1996). In words with an odd number of letters, there are more blanks than letters, and one-letter words are not blanked. For example, the cloze passage in section 14.5.1.1 would look like this as a C-test:

Many able students are not very proficient at L1>FL translation. They disc____ that, alth____ they ha__ good unders_______ of t__ target lang____ text, th____ simply can___ find t__ appropriate Eng____ in wh___ to exp____ the for____ language. T____ reasons f__ this c__ vary gre____, but i____ may sim____ be d____ to th____ knowledge o____ L1. I____ seems t____ be  partic____ true o____ able lear____ who ha____ learnt t____ FL naturali____, with min____ reference t____ their mot____ tongue. I____ such ca____, the for____ language i____ processed i____ ways mo___ akin t____ the proce____ of t____ mother ton____.

C-tests are an especially effective assessment instrument in inflected languages where key grammatical markers occur at the ends of words. They can also be effective in teaching all languages as they serve to stimulate discussion amongst students about what words might fit (paradigmatic knowledge), or how words are spelled and what part of speech would fit a given context (syntagmatic knowledge). C-tests use several texts (usually five) and so a large number of items are generated and texts are sampled in a more  representative way than with a cloze test. Their other advantage over cloze tests is that they promote processing of language both across clauses in a single sentence and across sentences. 
Critics of C-tests point out that they are not easy to devise and can take a long time to get right. It is also the case that they have poor ‘face validity’, ie students often react negatively to them and may not take them seriously, since the tests seem not to be related to language use. Proponents of C-tests, however, consider them a more valid means of testing general or overall competence, and the results of research into their use do suggest much higher levels of reliability and validity than can be achieved with cloze tests. 

Activity 25

There is only one way to learn about cloze and C-test procedures and that is to try them for yourself. To start with, limit yourself to a single C-test, and follow these steps:

1.	Find a text of appropriate difficulty for a language group you teach. 
2.	Follow the blanking procedure explained above and edit your text thoroughly to check you have done it accurately.
3.	Try it out on a few native speakers of the target language to see if any problems need ironing out or whether there are any ambiguities.

Now try it out on your students and see in what ways you can exploit it for teaching purposes. Do you think the task has ‘positive washback’ on teaching? (see Module 13, section 13.1.4.5). 

14.5.2  Assessing translation

Activity 26

Before tackling this section, have a look at the following quotation and reflect on the questions below:

One of the longest-running curses of language teaching and testing is that we, as teachers and testers, tend to do unto others what we, as pupils and students, had done unto us. (Stevenson, 1985: 138)

To what extent does your current use of translation reflect the way it was used when you were learning the FL? 
Are there any substantial differences? 
How were you tested on your language programme? 
What role did translation play?

L1>FL translation has traditionally been used to assess the candidate’s ability to use the FL accurately, especially with regard to grammar and lexis, while FL>L1 translation has normally been used to assess understanding of the FL and the ability to express it in appropriate English. Notwithstanding these deeply ingrained habits, it must be acknowledged that, in the view of most language testing specialists, translation is not an acceptable instrument to measure language proficiency. Although it is relatively easy to prepare and it has high face validity as a test, ie it seems to most of those who use it to be a suitable test, there is little research to suggest it correlates with other measures such as speaking, reading or writing. Translation seems to owe its place of pre-eminence in university language assessment largely to tradition and intuition.
	Furthermore, it is not always the able linguist who proves to be the good translator:

[One argument] runs somewhat like this: ‘Only the person who is highly proficient in a foreign language can produce a good translation. Therefore translation is a good technique for measuring L2 proficiency.’ The trouble with this reasoning is that although the first proposition is true, the second need not be. If it were true, then the best translations ought to be produced by the most highly proficient individuals, and the most highly proficient individuals are bilinguals. However, […] many bilinguals have considerable difficulty in translating from one language to another. Furthermore, as has been noted by numerous practising teachers, many students who show a high level of proficiency in other L2 production tasks tend to fall off badly when they are asked to translate. 
(Klein-Braley and Smith, 1985: 158)

Translation is a different skill from the other language skills. It involves subtle distinctions to be made in terms of both lexis and syntax. It requires very highly developed language skills and is certainly not for lower-level learners. 
	
Activity 27

1.	In what sense can translation be considered a ‘communicative’ activity? 

2.	Do you think this aspect of translation is as important as its role in testing linguistic accuracy?

Many now believe that translation in universities should be seen as a way to  train translators, rather than a vague means to support language acquisition. Module 11 presents ways in which the teaching of translation can be developed in this direction, emphasizing the communicative role of translation, its function as cultural mediation and what developing translators can learn from real-world translation activity. 
With regard to assessment of translation as a communicative vehicle, marking needs to be concerned with the extent to which the target text:
•	conveys the function of the source text;
•	reflects the form or text type of the source text;
•	observes appropriate linguistic conventions;
•	observes culture-specific conventions;
•	is coherent;
•	observes the rules of the language system.
(See also Module 11, section 11.5, on the use of translation in assessment.) On this view of translation, language accuracy thus becomes just one of several assessment criteria.
	Nevertheless, it is a fact that translation still figures very prominently in universities as a test of proficiency, and we need to consider how to ensure as high a degree of validity and reliability as possible. I would suggest that content validity can be achieved in part through the following: 
•	translation texts should be on topics covered in the module;
•	they should provide a clear context (ie students should be told the source of the text);
•	there should be a clear purpose (ie students should know why the text  was written and for whom);
•	texts should cover a range of registers and styles to reflect the texts encountered on the course.
Reliability can be best achieved through the use of clear criteria. One way to structure these is to consider, first, how well the student has understood the source text (ST), and secondly, how well he/she has rendered it in the other language. For example:

Grade	Understanding of ST	Rendition of ST
	
A (100-70%)	Demonstrates full understanding of the ST, including nuances; good understanding of cultural background	Translation captures meanings and nuances of ST; reads like idiomatic English; demonstrates sophisticated command of English lexis, syntax and register. No translationese. Virtually error-free.	
B (69-60%)	Demonstrates relatively full understanding of the ST; some appreciation of nuance and cultural background. Difficulties in lexis rather than grammar/syntax.	Shows some ability to find appropriate English expressions and structures, but suffers from occasional translationese (esp. in lexis). Not always sensitive to register. Guessed items suggest local and general context is fully understood.	
C (59-50%)	ST has generally been understood; but some weak stretches; difficulties in following syntax.	Adequate though not particularly idiomatic English; generally follows the sense of the ST, though without much attention to nuance and register. Guessed items suggest context is generally understood; most spelling rules followed.	
D (49-40%)	Substantial portion of ST not understood; failure to understand complex syntax; grammatical categories misconstrued.	Extensive interference from German lexis and syntax; translation reads awkwardly; sections of ST omitted or mistranslated. Guessed items show poor understanding of context; spelling deficient in places.	
Fail (39-0%)	At least half the ST not understood; inadequate knowledge of lexis, grammar (esp. syntax), and cultural background.	Substantial portions of translation do not make clear sense/large gaps left.  Very unidiomatic English; would not be readily understood by a reader.  Dominance of translationese; consistently poor spelling.	

NB: ‘Translationese’: a stretch of English text heavily influenced by the form of the ST, which a reader with no knowledge of German would not write.

(University of Birmingham, Department of German Studies)

Since holistic assessment of translations is very difficult to implement fairly, marking also probably ought to feature some more analytic or objective/mechanical system (see sections 14.2.2 and section 14.2.3). There follow five examples of such a system:

1.	Break the text into paragraphs or sections and allocate a number of points to each paragraph; deduct marks for errors from each sub-total (eg 1 for linguistic inaccuracy, outright mistranslation, or functionally or culturally inappropriate formulation; ½ for minor inaccuracies, false nuances or stylistic blemishes) up to a maximum of the sub-total. This approach will ensure that if the text you have chosen is uneven in difficulty, students will not be unduly penalized for one or two particularly weak sections.
2.	Apply the same procedure across the whole text, ie without creating sub-sections. Total the number of errors and then rank students. Unless you have used the texts before and have agreed mark boundaries relating to error totals, the danger here is that you will slip into norm-referencing by giving marks of 70+ to, say, the top 10%, 60+ to the next 25%, and so on.
3.	Build in a parallel positive marking scheme to the above, whereby bonus marks are allocated for effective translations or even just for accurate translation of difficult sections. To avoid excessive subjectivity, these need to be specified in the mark scheme before marking proper begins.
4.	In FL>English translation: identify, say, 20 anticipated problem areas (usually phrases or short clauses) in the FL text, based on your experience of having taught the course and on your knowledge of the students. Allocate the translation of each phrase a mark according to the following table:

excellent translation	5	
good translation	4	
correct translation	3	
inaccurate translation	2	
Grave error, distortion of original or not translated	1	
		 
Thus, if a student produced 2 inaccurate translations, 13 correct ones, 3 good versions and one excellent rendering, the score would be 60 (4+39+12+5). In addition, award an overall impression mark for the quality of the English version. A ratio of 2:1 in favour of the accuracy mark might be an appropriate balance. 

14.5.3  Using grammar tests

Activity 28

1.	Do you think it important to test grammar separately from other aspects of language?

2.	Does it make any difference if you are teaching near beginners or advanced learners?

Grammar tests measure one of the abilities underlying the various language skills. Some nowadays think that to test just one of these underlying abilities is inadequate, since on its own it cannot accurately predict mastery of the superordinate skill. Furthermore, testing grammatical structures in isolation, without any reference to their application to express meaning, can have a negative washback effect. It has also been suggested that grammar sub-tests correlate highly with other tests, especially reading (Alderson and Banerjee, 2002: 90), probably because grammar is assessed implicitly when superordinate language skills are being assessed (eg under the rubric ‘accuracy’ in writing or speaking test criteria). This is why many language exams these days do not include any specific grammar, or indeed vocabulary, component.
	However, especially at lower and intermediate levels, there are good reasons for including a specific assessment of grammar. Testing writing directly (as opposed to the indirect testing of grammar – see section 14.1.2 on direct versus indirect testing) makes it very difficult to cover most of the syllabus, since there is a clear limit on the areas (topics, registers, forms) we can get students to cover in an exam: any free writing task is a very selective snapshot of a student’s ability and we cannot be sure of how representative it is. Grammar tests, by contrast, can cover a lot of ground in the same amount of test time.
	Testing grammar can take various forms, most of them familiar from teaching, and indeed most of the grammar exercises featured in FL textbooks can be adapted for testing purposes.
Grammar testing is particularly important in placing students at appropriate levels or in suitable groups. Similarly, diagnostic testing of grammar can be helpful where doubts exist about the level of knowledge acquired in school education. Computer-based testing can be especially  useful here, saving time and being more manageable during the busy start-of-year period (see also 13.3.4). The DIALANG suite of tests (DIALANG, 2002) is especially to be commended. This is a substantial EU-funded project that aims to diagnose language competence in 14 languages. It offers assessment in listening, reading, writing, structures and vocabulary, and makes extensive use of both placement and self-assessment tests. It is possible to download a Pilot Tool and a Pedagogical Pack (http://www.dialang.org/english/pedapack).

14.5.4  Projects

Activity 29

Consider the following statement: “Projects are a good way of cutting back on teaching time. They are economic and fairly easy to mark.”  Would you agree with this view?

There is a lot of general information available on setting, supervising and marking student projects and dissertations (see, for example Race, 2001: 76–78; Marshall, 1999; Henry 1994). This section will highlight a small number of important points which are especially relevant to languages:
1.	Since the project or dissertation is likely to be the largest piece of FL writing students will undertake, it and they need thorough preparation.
2.	Students should have as much freedom as the course permits to choose their own topic, since they are then more likely to engage productively with the material they use.
3.	Be sure to review potential sources with students before they start and indicate those which are more accessible linguistically. The weakest language students, in particular, tend to struggle in their use of resources, often leading to plagiarism (see Module 13, section 13.1.9). 
4.	It is very good preparation to show students samples of how they should integrate FL source material without plagiarizing it, and how they should acknowledge and reference sources. Though this is a general academic skill, it is often not given sufficient attention when preparing for FL projects since students (and tutors) are more concerned about linguistic issues.
5.	Students will need support along the way, so, if possible, have intermediate deadlines – eg first for a plan of the project, then for the first draft of section 1, and so on. This will ensure students are proceeding along the right lines and are integrating source material appropriately with their own language, thus avoiding large-scale plagiarism which it is difficult to do much about at final-draft stage. 
6.	Arrange tutorial slots for students to discuss the project; or, if the student is abroad, arrange a regular e-mail exchange. Experience suggests these need to be compulsory if you want to maintain contact with the majority of students.
7.	A good check on plagiarism, or on writing by friendly natives abroad, is to involve students in an oral presentation or interview linked to the project. It is a useful exercise to include here, perhaps by way of introduction, an element of reflection or self-evaluation by students on what they feel was successful in the process and where they encountered difficulties. It is especially important for such orals or vivas to feature broadly comparable questions and linguistic complexity, so that, for example, students who have chosen a challenging topic are not disadvantaged by the ensuing linguistic complexity.
8.	In marking a project, the same applies as to all other assessment tasks: make sure criteria are clear, shared with students and understood by them. It is especially important that projects do not just assess those aspects of FL writing that other parts of the course assess, but also features that are specific to project work, such as information retrieval, exploitation and adaptation of FL resources, citation and referencing.
9.	Finally, be careful that you do not overvalue a piece of fluent and accurate FL writing that is very weak in other areas. This is where sub-categories in criteria are important; such headings as quality of ideas, originality of thinking, coherence of argument, etc, need to be assessed individually. Conversely, don’t allow a morass of linguistic error to blind you to positive features of content. This applies to any FL essay, of course, but is a particular danger on longer pieces of writing where the cumulative effect of  errors can be quite depressing.

14.5.5  Portfolios
A portfolio is a collection of work produced by a student that demonstrates  achievements, progress made, as well as effort invested in the task. The student usually decides what is to be included as evidence and provides an accompanying self-reflective piece of writing. Students may sometimes also be involved in helping to formulate the criteria for assessing the portfolio.
	From the student’s point of view, the variability of what one might consider acceptable output is probably the greatest strength of the portfolio. Depending on the topic and level, it might include:

•	graphics and pictures;
•	learner diaries, correspondence;
•	marked work;
•	a list of learning materials used;
•	the results of Internet searches;
•	reports on activities, surveys or investigations undertaken in the target country (maybe during residence);
•	reflection on issues, etc. 

With lower-level language tasks, one might include the completed tasks themselves, reflection on perceived weaknesses and an indication of remedial action taken.
	The advantages of portfolios are that they show how a student has developed linguistically and they are thus more informative than one-off or ‘snapshot’ language assessments. They also show if and how students have responded positively to feedback.
	On the other hand, a major problem with portfolios is that one can never be sure that all the work is the student’s own. To address this concern, it may be necessary to do a viva in the FL linked to the portfolio. Furthermore, the individualized and therefore extremely diverse nature of portfolios can make it very difficult to formulate reliable marking criteria, especially if large numbers of students are involved. Indeed one might question to what extent anything that is deliberately intended to be unique to a student is amenable to fair and just assessment using fixed criteria. While they can have huge motivational benefits and provide very positive washback on classroom learning, portfolios are probably not ideal as the exclusive instruments of summative assessment, but they may well have an application in assessing residence abroad. For ideas on designing portfolios and issues to consider when marking them, see Baume (2001) and Race (2001: 68–70).
	An important development in modern languages is the European Language Portfolio which aspires to promote democratic citizenship and mobility in Europe by recording the language knowledge and experiences of the holder. It consists of three parts:

1.	a ‘passport’ section recording contact with language qualifications;

2.	a personal language biography detailing contact with foreign languages and cultures;

3.	a dossier with evidence of linguistic competence.

Comparison of language qualifications is achieved through reference to the levels of the Common European Framework of Reference (Council of Europe, 2001). For further details on the Portfolio, see Little and Perclová (2001) and Gauthier (2002).

14.5.6  Reviews
As in undergraduate teaching generally, reviewing FL books is an excellent way to get students to read both actively and critically, and to employ the key cognitive skills of selecting, comparing and evaluating. Reviews need not be limited to books, but might also profitably encompass films or even TV programmes. The activity can begin at lower levels of proficiency, with students reviewing adapted or easy FL readers, or subtitled films. Students of literature at advanced level might review short set texts or other works by set authors; alternatively, reviews of long FL articles might be appropriate in some circumstances. By final year, all students should be capable of reviewing single books or full-length films on a variety of topics. 
Whether the review is to be written in L1 or the FL will clearly depend on students’ level of proficiency and the purpose of assessment (is it a form of reading or writing assessment, or both?). Generally, by the second year of a post-‘A’ level course, reviews should be in the FL, by which stage students should be adopting the relevant conventions of reviewing, after having considered sample reviews that have appeared in FL publications.
	Issues to consider when introducing FL reviews include:

•	show students sample reviews and the degree of summarizing versus evaluation that they feature;
•	get students to work within a word limit;
•	put students’ reviews on the departmental website or maybe include a couple of samples in the following year’s module handbook;
•	design appropriate criteria; these are likely to vary depending on the level of FL proficiency;
•	ensure you, the department, language centre or library have sufficient resources to sustain the activity;
•	if appropriate resources are in short supply, ask students to review the same text or film from different perspectives – eg a pamphlet on French education might be reviewed for different target audiences: teachers, parents or foreign readers.

14.6  Conclusion
The key lesson to be learnt about all assessment, not just language assessment, is that testing should be closely linked to the learning outcomes of our courses. This means being very precise about what it is we want to measure and selecting a suitable assessment form (eg exam versus portfolio or project) and an appropriate test type (grammar, reading, mixed-skill, etc). None of the tasks described in this module is without its drawbacks; there is, in fact, no such thing as the perfect language test, and the best we can do is to ensure that our chosen instrument of assessment meets as closely as possible the five key criteria of validity, reliability, transparency, practicality and washback. 
	Just as  importantly, we know from, among other things, research into learning styles (see Module 12) that not all learners respond well to the same mode of learning or testing. We are therefore most likely to be fair to students and to allow them every opportunity to show their ability, if we ensure our assessment procedures feature a range of methods and tasks. This means combining communicative tests (eg free writing) with those that focus on sub-skills (eg grammar exercises) and balancing subjectively marked tasks (eg oral interviews) with more objective ones (eg completion of a negotiation task).

Activity 30
The following is a self-assessment exercise on this module. Referring back, as necessary, to the relevant section in the right-hand column, write a short paragraph about each of the following statements:

	Section	
The only tests university language tutors need to be concerned with are achievement tests. 	14.1.1  	
Sound testing systems feature indirect testing of underlying elements of language, such as pronunciation, grammar or vocabulary.	14.1.2 	
Language testing should always deal with one skill at a time.	14.1.2	
Integrative skills testing is not sufficiently reliable for use on university language courses.  	14.1.3	
Of the three methods of marking student work, the analytic and objective approaches have most to recommend them.	14.2.1-3
	
There is really only one valid approach to the objective marking of FL essays.	14.2.3	
Advanced-level university students do not need much help with writing FL essays, other than vocabulary. By this stage, they should know how to tackle discursive titles. 	14.2.4	
If used for assessment purposes, student oral presentations should always be an individual task.	14.3.2	
There are basically three types of oral exam question. 
	14.3.3	
The most effective listening and reading comprehension tasks are open-ended questions. Multiple-choice or true / false questions have no place in university language testing. 	14.4.1, 14.4.2	
We should be aiming to banish English completely from our testing of foreign languages.
	14.4.4	
Cloze tests are both a reliable and valid form of language testing.	14.5.1	
C-tests are both a simple and economic  form of assessing overall language competence.	14.5.1	
Assessment of translation work needs to focus on linguistic accuracy.	14.5.2	
The danger of plagiarism makes the use of projects and portfolios in language work too problematic.	14.5.4, 14.5.5	

14.7  Commentary on activities

Activity 2

Test description	Test type	
These assign students to appropriate levels within a programme; such tests are related to course objectives.	Placement	
Such tests assess students’ general ability to learn languages prior to actual tuition in any particular language.	Aptitude	
Tests used for screening purposes to make decisions about students’ ability to cope with a course.	Admissions	
Exams to assess FL speakers’ ability in the target  language independent of any curriculum.	Proficiency	
Tests used during a course to provide information about students’ mastery of or problems with elements of the syllabus.	Progress	
These identify students’ existing strengths and weaknesses in order to help tailor tuition to learners’ needs.	Diagnostic	
These assess what students have accomplished in relation to a particular course of study or module. They usually occur at the end of a course.	Achievement	

Activity 7
The obvious things you would need to know are the type of degree programme these students are on, the expectations or learning outcomes of the course, their previous learning experiences, what the task was, how it fits into the course, what the marking culture is, the marking scales employed and, of course, some marking criteria. These things seem pretty obvious perhaps and the task even seems a bit absurd, yet it is not terribly different from the one our universities set incoming foreign language assistants every year, often with minimal induction and support. The chances are that if you just dived in and marked the French student’s essay according to your own rough expectation of such a piece of work, your marks would be wholly inappropriate for the particular context.
	The point is that assessment is not an absolute science, and so much depends on the context in which the assessment takes place. Even closer to home, the same essay set on one UK university first-year course might be set on another university’s final year and the range of marks attained might be very similar. This is because, as Coleman (1996) discovered, the entry-level foreign language proficiency in certain universities exceeds the exit-level proficiency at some others. This does not mean, however, that in the respective contexts the two assessments cannot be valid instruments of assessment. They can. What matters is the existence of good criteria to ensure marking in each case is reliable and fair.

Activity 8
Clearly Letter 1 displays a number of linguistic weaknesses (tenses, adverbs, poor lexical choices, some orthographical problems). However, it employs an appropriate format, is well laid out, addresses the task, refers to the criteria for the job and does everything asked. 
Letter 2, by contrast, is very accurate, and displays breadth of vocabulary and a command of complex sentence structure. However, it is badly organized, a little short, fails to address the question properly, uses inappropriate formulae and adopts the wrong register in places. 
	Depending on the criteria you employed and the weighting you gave to language versus content, completion of task and appropriateness of formats, these might both have come out with average to poor marks for the target level.

Activity 11
Holistic approaches are generally valid, highly practical, can ensure high reliability across markers and emphasize student strengths. While the latter may suggest some minimal sort of positive washback on teaching, feedback to students is very limited and the broad, undifferentiated categories do not provide much information to students or tutors on areas that need to be worked on, thus also limiting the transparency of the approach.
Analytic approaches are highly valid. The use of marking grids can make marking quite practical and a reasonably high marker reliability can be achieved through sample marking in teams and second- or double-marking procedures. By breaking performance up into its constituent elements, they make marking criteria explicit, allow specific, targeted feedback on weaknesses and provide tutors with key information on students’ abilities in different areas, both of which ensure very positive washback on learning and teaching. 
Objective marking is probably the least practical of the three owing to its time-consuming arithmetic and the need to link it to parallel analytic marking of, for example, organization and content. However, it is probably the most valid, at least on accuracy of student production, ensures very high marker reliability, is completely transparent and provides clear (if negative) feedback for students. Generally, its washback effect is positive.

Activity 12
1.	This places an undue burden on imagination. Apart from diluting the main purpose, ie to assess a student’s ability to write in the FL, this disadvantages the student with little imagination or the student whose mind goes blank in the exam room.
2.	This favours those students with a certain amount of creativity and especially those who are able to develop arguments on almost any topic and to present them in a logical fashion. It might be argued that this is an element of the essay task, but it disadvantages those who are less creative. It is also a task most would struggle to perform in L1, let alone an FL.
3.	This requires background knowledge of the topic. We are not interested in students’ general knowledge but in their FL writing ability. The title might be appropriate if students were given some ideas to work with, which would ensure a slightly more level playing field. 
4.	This too depends on general knowledge but, worse than this, it requires in addition a knowledge of economic issues, which not all students may possess. If the topic has featured in coursework, it may be fair enough; otherwise avoid it.
5.	Apart from the inauthentic nature of the task in providing spoken language in writing, this again relies a lot on creativity under pressure and is a test of script-writing ability as much as anything.

Activity 13
Criteria for a task such as this need to take account of how well the task has been performed, the relevance of the information provided and/or given, the appropriateness to the situation of the language used (lexis and register), as well as the usual criteria for oral assessment, such as accuracy of language and the quality of pronunciation and intonation. Criteria for role plays are difficult to write; often they need to include slight variations to recognize the demands of different role-play situations. Here they would have to address the distinctive challenges of the two positions students are placed in: one in which they might need to placate, explain and justify, and the other in which they might need to apply skills of persuasion.

Activity 19

Test format
	Student task	Exercise type	
Spoken text and clock faces	Enter times heard onto clockface	GR	
Picture and oral statements about it	Decide whether each statement is correct 	TF	
Spoken text and map with key to symbols	Put symbols on map in relation to what you hear	GR	
Several pictures and one oral statement	Tick the appropriate picture	MC	
Spoken text and written L1 statements	Decide whether each statement is correct	TF	
Spoken questions and several written answers on each	Tick the appropriate answers	MC	
Spoken text and written key words relating to it	Tick which of the ideas occurs in the text	TF	
Spoken text and L1 questionnaire	Write in L1 words on questionnaire	OE	
Spoken text and street map	Transfer data to street map	GR	
Spoken text and several oral statements	Tick the appropriate statement	MC	
Pictures and oral statements	Decide which pictures go with which statements	OM	
Spoken text	Draw an object described in the text	GR	
Spoken text and oral statements about it	Decide whether each statement is correct	TF	
Spoken text and jumbled pictures	Put pictures in correct sequence	OM	
Spoken text and several written L1 statements 	Tick the appropriate statement	MC	
Spoken numbers or dates	Write down figures	GR	

Activity 20

Text type	Exercise type	Type of reading	

Multiple points of view (eg questionnaires, interviews)	
Ordering/matching	
Gist and detailed reading	

Lengthy, self-contained texts (eg stories, reports, factual articles)	
(1) Open-ended

(2) Multiple-choice

(3) True/false	
Gist, detailed, skimming and scanning

Detailed

Global, detailed	

Texts read quickly for specific information (eg timetables, TV guides)
	
(1) Ordering/matching

(2) Open-ended
	
Skimming, scanning

Skimming, scanning	

Activity 21

Test format	Student task	Exercise type	
Picture with written statements	Tick the correct statement(s)	TF	
Written texts and beginnings/endings of sentences	Decide which sentence parts go together in relation to the text	OM	
Questions on a text with, in each case,  alternative answers	Tick the correct alternative	MC	
Several pictures and one written statement	Tick the appropriate picture	MC/OM	
Descriptions of people and key words about them	Write down the names of the people associated with the key words	OM	
Written text with written statements	Tick the correct statement(s)	TF	
Written text with written statements	Tick the appropriate statement	MC/OM	
Written text and accompanying table	Tick the facts mentioned in the text	OM	
Written text	Underline sections of text important to a specific question (eg ‘arguments for’)	OE	
Written text	Draw a plan of something described in the text	GR	
Written texts and pictures	Link pictures to the relevant texts	OM	
One picture and several written statements	Tick one of the statements	MC/OM	
Written texts and headings	Link headings to the appropriate texts	OM	
Questions and answers	Match the questions with the answers	OM	
Written text with map and key to symbols	Put symbols on map in relation to content of text	GR	
Written text	Summarize main contents in L1	OE	
Written text and jumbled sentences about it	Put sentences in correct order	OM	
Introductory sentence or clause with several possible continuations	Tick the appropriate continuation	MC	
Written text and L1 questions on text	Answer questions on text content	OE	
Jumbled phrases/clauses	Put phrases/clauses in correct order	OM	

Activity 24
The full text is: 
Many able students are not very proficient at L1>FL translation. They discover that, although they have good understanding of the target language text, they simply cannot find the appropriate English in which to express the foreign language. The reasons for this can vary greatly, but it may simply be due to poor knowledge of L1. It seems to be  particularly true of able learners who have learnt the FL naturalistically, with minimal reference to their mother tongue. In such cases, the FL is processed in ways more akin to the processing of the mother tongue.

There are several potential ambiguities here: item 1 could be a range of epithets (‘excellent’, ‘outstanding’, ’perfect’, etc); one might argue for ‘with’ in item 3; item 5 might be ‘yet’, ‘however’, etc; while item 11 might also be ‘circumstances’. This illustrates how careful one has to be in devising a cloze test. The one above would either need a comprehensive mark scheme or it would need editing to avoid the ambiguities.

14.8  Appendices

Appendix 1

Criteria for assessing short essays

University of Birmingham, Department of German Studies

Name: ______________________________

	A (100-70%)	B (69-60%)	C (59-50%)	D (49-40%)	
					
Facts
and
Ideas
	Many good and appropriate ideas; some originality; facts correct.	A range of good ideas; facts appropriate.	Some good ideas, mostly appropriate; facts mainly correct.	Few good ideas, not all appropriate; some important facts incorrect.	

Argumentation
and Communication	Ideas linked; coherent argumentation; effective at intellectually complex level.	Most ideas linked; argumentation reasonably demanding.	Some ideas linked; argumentation simple and not always effective.	Few ideas linked; coherent argumentation generally lacking.	

Grammar

	Predominantly accurate; wide range of structures enhance topic.	Reasonably accurate; wide range of structures attempted.	Fairly sound; mainly simple structures; most spelling rules followed.	Shaky; simple structures; some intrusive errors; spelling deficient in places.	

Sophistication	Very broad range of vocab. used to good effect; register appropriate; attempted level complex.	Good range of vocabulary; register often correct; attempted level appropriate.	Limited range of vocabularly with little attempt to use appropriate register.	Insufficient, impedes presentation of topic; no attempt at complexity or appropriate register.	

*	The use of unattributed material as if it is the student’s own intellectual property may result in failure of the essay as a whole and a mark of zero.

											Overall Mark:_______________________

 
Appendix 2

Criteria for assessing student oral presentations

University of Birmingham, Department of German Studies

Name:________________________________

	A (100-70%)	B (69-60%)	C (59-50%)	D (49-40%)	
					
Pronunciation and
Intonation	Accurate, varied; no noticeable mother-tongue patterns.	Accurate and varied; very few mother-tongue patterns.	Acceptable.  Some flaws; attempts to vary patterns; no undue strain on listener.	Flat intonation; poor pronunciation; listener has to work hard to follow.	
Quality
Of
Language	Minimal formal errors; wide range of structures.	A few formal errors do not distract from content; good range of structures.	Several formal errors but rarely interfering with communication; some attempt to vary structures.	Frequent formal errors; communication disrupted; little structural variety.	

Quality
Of
Presentation*	Excellent use of paralinguistic features/visual aids; minimal reference to notes; good steady pace; audience orientated.	Good use of paralinguistic features/ visual aids; appropriate use of notes; easy to follow.	Paralinguistic features enhance overall performance; heavy reliance on notes, but some audience awareness.	Little awareness of audience;  paralinguistic features/visual aids poor.  Whole sections read out - pace inappropriate.	

Content
	Comprehensive coverage; entirely relevant; cogent.	Wide coverage; most points covered in depth; relevant examples; well structured.	Reasonable coverage; relevant; some depth to points made.	Information largely correct and relevant, but superficial.	

Independence
	Exploitation of sources shows extensive ability to reformulate, summarize, expand; questions fully answered.	Some ability to reformulate, summarize and expand; all questions well answered, if not always in depth.	Reasonably independent communication;  dependence on verbatim source material; answers repeat parts of talk.	Content and/or language relies substantially on verbatim use of source material; answers to questions patchy or incomplete.	

Handout	Comprehensive, well structured and laid out; audience’s task easy; extensive linguistic support and error-free.	Full plan with appropriate detail; considerable help for listener; good vocabulary support; very accurate.	Good plan with some detail;  listener needs to fill in several gaps; some linguistic support; largely accurate.	Brief outline of points; listener has to do a lot of work to follow; minimal linguistic back-up; several inaccuracies.	

* Paralinguistic features include: body language, gestures, facial expressions and eye contact.
		Overall Mark:_______________________

Appendix 3

Marking criteria for written work

University of Birmingham, Centre for Modern Languages

Three elements are taken into account: 	Accuracy (50% of total mark)
					Range of expression (30% of total mark)
					Communication of content (20% of total mark)
	
Material which is completely irrelevant to the task set can score a maximum of 40%

First a mark out of 20 is established. This is then multiplied by 5 to obtain a percentage. 

Scale	Accuracy	
9-10	Student shows very good ability to use appropriate structures and with very few errors. The work is clear and easy to read.	
7-8	Good assimilation of material covered, still a few errors but no basic ones (eg in French: verb endings, agreements).	
4-6	Quite a few errors showing that the work covered has not been assimilated properly but the message is comprehensible.	
2-3	Frequent basic errors which may impede comprehension of the message.
	
0-1	Work almost incomprehensible, word order completely wrong, words not recognizable.	

Scale	Range of Expression	
5-6	Very good range and variety of vocabulary, sentence structure and links.
	
3-4	Vocabulary usually adequate for the task, though some mother tongue interference. Student shows ability to put the work covered into practice.  Good linking of sentences.	
2	Very limited vocabulary. A lot of repetitions. Mostly very simple structures used.  Difficult to understand.	
0-1	Severely limited vocabulary. Only the most simple structures used.

Scale	Communication of Content	
3-4	The work is well structured and relevant to the task set. If it is a letter, it has an appropriate beginning and ending. If it is an essay, it has a good introduction and conclusion and the development of ideas is clear. Each idea is developed and not just quoted.	
1-2	There are some problems with the structure and the ideas are not always well developed or relevant to the title. The student does not respond fully to the task set.	
0	Very little relevant information conveyed, no development of ideas, no introduction, no conclusion.	

14.9  References

Alderson, C J (2000) Assessing Reading, Cambridge University Press, Cambridge
Alderson, C J and Banerjee, J (2001) Language testing and assessment. Part 1, Language Teaching, 34 (4), 213–36
Alderson, C J and Banerjee, J (2002) Language testing and assessment. Part 2, Language Teaching, 35 (2), 79–113
Atkinson, T (2002) Computer aided assessment and language learning, ICT4LT, http://www.ict4lt.org/en/ (module 4.1)
Bailey, K M (1998) Learning About Language Assessment: Dilemmas, decisions and directions, Heinle & Heinle, Boston
Baume, D (2001) A Briefing on Assessment of Portfolios, LTSN Generic Centre Assessment Series, LTSN, York
Bolton, S (1996) Probleme der Leistungsmessung. Fernstudienprojekt ,Deutsch als Fremdsprache und Germanistik‘. Fernstudieneinheit 10, Langenscheidt, Berlin/Munich
Coleman, J A (1996) Studying Languages. A survey of British and European students, CILT, London
Council of Europe (2001) A Common European Framework of Reference for Learning, Teaching and Assessment, Cambridge University Press, Cambridge
DIALANG (2002) http://www.dialang.org
Doyé, P (1988) Typologie der Testaufgaben für den Unterricht Deutsch als Fremdsprache, Langenscheidt, Berlin/Munich
Gauthier, V (2002) The ELP: Useful references for teachers, CercleS Bulletin, 15, 5–6
Grotjahn, R (ed) (1996) Der C-Test. Theoretische Grundlagen und praktische Anwendungen 3, Brockmeyer, Bochum
Henry, J (1994) Teaching Through Projects, Kogan Page, London
Hughes, A  (1981) Conversational cloze as a measure of oral ability, ELT Journal, 35 (2), 161–68
Hughes, A (1989) Testing for Language Teachers, Cambridge University Press, Cambridge
Klapper, J (1991) The role of the video camera in communicative language teaching and assessment, Language Learning Journal, 4, 12–15
Klein-Braley, C and Raatz, U (1984) A cloze-up on the C-Test: A study of the construct validation of authentic tests, Language Testing, 2 (1), 76–104 
Klein-Braley, C and Smith, V (1985) ‘Incalculable and full of risks?’: translation L1 to L2 as a testing procedure, in Translation in Foreign Language Teaching and Testing, ed C Titford and A E Hieke, pp 155–68, Gunter Narr Verlag, Tübingen
Lado, R (1961) Language Testing, Longman, London
Little, D and Perclová, R (2001) European Language Portfolio Guide for Teachers and Teacher Trainers, Council of Europe, Strasbourg
McNamara, T (2000) Language Testing, Oxford University Press, Oxford
Marshall, S (1999) Supervising projects and dissertations, in A Handbook for Teaching and Learning in Higher Education, ed H Fry, S Ketteridge and S Marshall, pp 108–19, Kogan Page, London
Neather, E J, Woods, C, Rogrigues, I, Davis, M and Dunne, E (1995) Target Language Testing: A report of a project on the testing of reading and listening without the use of English, School Curriculum and Assessment Authority, London
Oller, J W (1979) Language Tests at School, Longman, London
Page, B (1993) Target language and examinations, Language Learning Journal, 8, 6–7
Powell, B, Barnes, A and Graham, S (1996) Using the Target Language to Test Modern Language Skills, School Curriculum and Assessment Authority, London
Race, S (2001) The Lecturer’s Toolkit: A practical guide to learning, teaching and assessment, 2nd edition, Kogan Page, London
Skehan, P (1998) A Cognitive Approach to Language Learning, Oxford University Press, Oxford
Spolsky, B (1995) Prognostication and language aptitude testing 1925–62, Language Testing, 12 (3), 321–40 
Stevenson, D K (1985) Language testers and translation tests, in Translation in Foreign Language Teaching and Testing, ed C Titford and A E Hieke, pp 137–53, Gunter Narr Verlag, Tübingen
Underhill, N (1987) Testing Spoken Language: A handbook of oral testing techniques, Cambridge University Press, Cambridge
Weir, C J (1990) Communicative Language Testing, Prentice Hall International, Hemel Hempstead

14.10  Recommended further reading

Alderson, C J and Banerjee, J (2001–02) Language testing and assessment. Parts 1 and 2, Language Teaching, 34 (4), 213–36, and 35 (2), 79–113
An authoritative two-part state-of-the-art review. Of interest to those wanting to pursue recent published research on any aspect of language testing.

ALTE (The Association of Language Testers in Europe) www.alte.org 
This is an association of 27 institutions within Europe, each of which produces examinations and certification for language learners. Each member provides examinations of the language which is spoken as a mother tongue in its own country or region. It covers six levels of proficiency, aligned with the Council of Europe’s framework (Council of Europe, 2001), and allows comparison of the levels of examinations in a wide range of languages. It also provides a ‘Can Do’ description in 13 languages according to levels and skill areas. 

Bailey, K M (1998) Learning About Language Assessment:  Dilemmas, decisions and directions, Heinle and Heinle, Boston
A highly readable book with good sections on general principles of assessment, but also specific chapters on cloze and C-tests, multiple-choice tests, the nowadays much-neglected dictation, as well as approaches to marking. Although written for a specific American audience, it is readily accessible to all.

DIALANG (2002) http://www.dialang.org
This is a set of computer-based diagnostic tests available over the Internet. It employs self-assessment ratings in conjunction with the results of objective tests to offer the user a test of an appropriate level of difficulty. The language used to administer the test can be chosen by the user from a list of 14 European languages, as can the skill to be tested: reading, writing, listening, grammar and vocabulary. Finally, DIALANG can provide immediate feedback on test scores, as well as  advice on how learners can improve their proficiency to the next level based on the Common European Framework (Council of Europe, 2001).

Hughes, A (1989) Testing for Language Teachers, Cambridge, Cambridge University Press
If you want to consult just one volume on the subject, this is it. All relevant issues are presented with admirable clarity. Jargon is kept to a minimum and practicality emphasized throughout. A must.

McNamara, T (2000) Language Testing, Oxford University Press, Oxford
A brief overview of the subject for the non-specialist which includes short surveys of different aspects of testing, uncluttered by references, readings from key texts and a selection of annotated references.

Oller, J W (1979) Language Tests at School, Longman, London
Although clearly not the latest word on assessment, this ground-breaking book put forward arguments in favour of the sort of integrative language testing that led ultimately to the introduction of communicative language tests. In particular, it introduced many in the language teaching world to the cloze test for the first time.

Underhill, N (1987) Testing Spoken Language: A handbook of oral testing techniques, Cambridge University Press, Cambridge
A comprehensive survey of the field which is bursting with practical suggestions. If you are ever short of ideas on assessing any aspect of oral work, this is the place to look for inspiration. Considers pros and cons of various test types, and looks at marking systems and the question of test evaluation.

Weir, C J (1990) Communicative Language Testing, Prentice Hall International, Hemel Hempstead
Besides offering a straightforward introduction to test design and test construction, this book illustrates all the main approaches to assessing the four language skills, listing the advantages and disadvantages of each.

The Cambridge University Press series on assessment in languages represents the most up-to-date and authoritative statement on the subject. However, these detailed books are not for the beginner but rather for those wanting to explore a particular skill area in greater depth. Volumes to have appeared to date are:

Alderson, C J (2000) Assessing Reading, Cambridge University Press, Cambridge (Chapter 7 on ‘Techniques for testing reading’ is the most useful for the classroom practitioner)

Buck, G (2001) Assessing Listening, Cambridge University Press, Cambridge

Cushing Weigle, S (2002) Assessing Writing, Cambridge University Press, Cambridge

Douglas, D (2001) Assessing Languages for Specific Purposes, Cambridge University Press, Cambridge

Read, J (2000) Assessing Vocabulary, Cambridge University Press, Cambridge

14.11  Assessment task

1.	Write a critique of your department’s assessment of language work. You should structure your critique around the following points:

•	The various purposes of testing in the department.
•	The extent to which different language skills are assessed.
•	How written work is tested and marked.
•	The part played by grammar testing.
•	The variety of oral assessment tasks that are employed.
•	How listening and reading are assessed.
•	The role of translation.

2.	Is there scope for expanding the range and variety of testing techniques employed? What would be the advantages of doing this? What problems would you envisage and how might they be overcome?