14.5.2
Assessing translation

Activity 26

Before tackling this section, have a look at the following quotation and reflect on the questions below:

One of the longest-running curses of language teaching and testing is that we, as teachers and testers, tend to do unto others what we, as pupils and students, had done unto us. (Stevenson, 1985: 138)

To what extent does your current use of translation reflect the way it was used when you were learning the FL?
Are there any substantial differences?
How were you tested on your language programme?
What role did translation play?

L1>FL translation has traditionally been used to assess the candidate's ability to use the FL accurately, especially with regard to grammar and lexis, while FL>L1 translation has normally been used to assess understanding of the FL and the ability to express it in appropriate English. Notwithstanding these deeply ingrained habits, it must be acknowledged that, in the view of most language testing specialists, translation is not an acceptable instrument to measure language proficiency. Although it is relatively easy to prepare and it has high face validity as a test, ie it seems to most of those who use it to be a suitable test, there is little research to suggest it correlates with other measures such as speaking, reading or writing. Translation seems to owe its place of pre-eminence in university language assessment largely to tradition and intuition.

Furthermore, it is not always the able linguist who proves to be the good translator:

[One argument] runs somewhat like this: 'Only the person who is highly proficient in a foreign language can produce a good translation. Therefore translation is a good technique for measuring L2 proficiency.' The trouble with this reasoning is that although the first proposition is true, the second need not be. If it were true, then the best translations ought to be produced by the most highly proficient individuals, and the most highly proficient individuals are bilinguals. However, […] many bilinguals have considerable difficulty in translating from one language to another. Furthermore, as has been noted by numerous practising teachers, many students who show a high level of proficiency in other L2 production tasks tend to fall off badly when they are asked to translate.
(Klein-Braley and Smith, 1985: 158)

Translation is a different skill from the other language skills. It involves subtle distinctions to be made in terms of both lexis and syntax. It requires very highly developed language skills and is certainly not for lower-level learners.

Activity 27

  1. In what sense can translation be considered a 'communicative' activity?

  2. Do you think this aspect of translation is as important as its role in testing linguistic accuracy?

Many now believe that translation in universities should be seen as a way to train translators, rather than a vague means to support language acquisition. Module 11 presents ways in which the teaching of translation can be developed in this direction, emphasizing the communicative role of translation, its function as cultural mediation and what developing translators can learn from real-world translation activity.

With regard to assessment of translation as a communicative vehicle, marking needs to be concerned with the extent to which the target text:

  • conveys the function of the source text;
  • reflects the form or text type of the source text;
  • observes appropriate linguistic conventions;
  • observes culture-specific conventions;
  • is coherent;
  • observes the rules of the language system.

(See also Module 11, section 11.5, on the use of translation in assessment.) On this view of translation, language accuracy thus becomes just one of several assessment criteria.

Nevertheless, it is a fact that translation still figures very prominently in universities as a test of proficiency, and we need to consider how to ensure as high a degree of validity and reliability as possible. I would suggest that content validity can be achieved in part through the following:

  • translation texts should be on topics covered in the module;
  • they should provide a clear context (ie students should be told the source of the text);
  • there should be a clear purpose (ie students should know why the text was written and for whom);
  • texts should cover a range of registers and styles to reflect the texts encountered on the course.

Reliability can be best achieved through the use of clear criteria. One way to structure these is to consider, first, how well the student has understood the source text (ST), and secondly, how well he/she has rendered it in the other language. For example:

Grade
Understanding of ST
Rendition of ST

A (100-70%)

Demonstrates full understanding of the ST, including nuances; good understanding of cultural background

 

 Translation captures meanings and nuances of ST; reads like idiomatic English; demonstrates sophisticated command of English lexis, syntax and register. No translationese. Virtually error-free.

B (69-60%)

 Demonstrates relatively full understanding of the ST; some appreciation of nuance and cultural background. Difficulties in lexis rather than grammar/syntax. Shows some ability to find appropriate English expressions and structures, but suffers from occasional translationese (esp. in lexis). Not always sensitive to register. Guessed items suggest local and general context is fully understood.
C (59-50%)
 ST has generally been understood; but some weak stretches; difficulties in following syntax. Adequate though not particularly idiomatic English; generally follows the sense of the ST, though without much attention to nuance and register. Guessed items suggest context is generally understood; most spelling rules followed.
D (49-40%) Substantial portion of ST not understood; failure to understand complex syntax; grammatical categories misconstrued.
 Extensive interference from German lexis and syntax; translation reads awkwardly; sections of ST omitted or mistranslated. Guessed items show poor understanding of context; spelling deficient in places.

Fail (39-0%)

At least half the ST not understood; inadequate knowledge of lexis, grammar (esp. syntax), and cultural background.


Substantial portions of translation do not make clear sense/large gaps left. Very unidiomatic English; would not be readily understood by a reader. Dominance of translationese; consistently poor spelling.

NB: 'Translationese': a stretch of English text heavily influenced by the form of the ST, which a reader with no knowledge of German would not write.

(University of Birmingham, Department of German Studies)

Since holistic assessment of translations is very difficult to implement fairly, marking also probably ought to feature some more analytic or objective/mechanical system (see sections 14.2.2 and section 14.2.3). There follow five examples of such a system:

  1. Break the text into paragraphs or sections and allocate a number of points to each paragraph; deduct marks for errors from each sub-total (eg 1 for linguistic inaccuracy, outright mistranslation, or functionally or culturally inappropriate formulation; ½ for minor inaccuracies, false nuances or stylistic blemishes) up to a maximum of the sub-total. This approach will ensure that if the text you have chosen is uneven in difficulty, students will not be unduly penalized for one or two particularly weak sections.
  2. Apply the same procedure across the whole text, ie without creating sub-sections. Total the number of errors and then rank students. Unless you have used the texts before and have agreed mark boundaries relating to error totals, the danger here is that you will slip into norm-referencing by giving marks of 70+ to, say, the top 10%, 60+ to the next 25%, and so on.
  3. Build in a parallel positive marking scheme to the above, whereby bonus marks are allocated for effective translations or even just for accurate translation of difficult sections. To avoid excessive subjectivity, these need to be specified in the mark scheme before marking proper begins.
  4. In FL>English translation: identify, say, 20 anticipated problem areas (usually phrases or short clauses) in the FL text, based on your experience of having taught the course and on your knowledge of the students. Allocate the translation of each phrase a mark according to the following table:
    excellent translation
    5
    good translation
    4
    correct translation
    3
    inaccurate translation
    2
    Grave error, distortion of original or not translated
    1

Thus, if a student produced 2 inaccurate translations, 13 correct ones, 3 good versions and one excellent rendering, the score would be 60 (4+39+12+5). In addition, award an overall impression mark for the quality of the English version. A ratio of 2:1 in favour of the accuracy mark might be an appropriate balance.

previous button
next button

contents button