Evaluation of English-Norwegian machine translation
An article about machine translation was translated into Norwegian by Google Translate (https://translate.google.com). In November 2009, professional translators evaluated the translation for fluency and for accuracy.
The results for fluency are in Figure 1. The results for accuracy are in Figure 2. Full details of the evaluation are in mt-evaluation-en-no.xls.
Figure 1. The fluency of Norwegian machine translation of international English
Figure 2. The accuracy of Norwegian machine translation of international English
Translators who evaluated the machine translations include the following people:
- T1, Karin Berling (www.translatorscafe.com/cafe/member95153.htm)
- T4, Jon Kindberg
- T5, Micha Reisel
Discussion of the results
Sometimes, translators gave different evaluations.
For some sentences, the difference is because our instructions were not sufficiently clear. We did not tell translators how to evaluate text that was not translated. For example, for some sentences, the technical term 'machine translation' was not translated. Translators evaluated these sentences differently. For accuracy, some translators said that if a translation contains English words, they evaluated the translation as 'partly correct'. Some translators said that if a translation contains English words, they evaluated the translation as 'nonsense'.
Sentence 38 is an example of what not to write. The translation is low quality.
Some comments from the translators
"It was a bit confusing to have both the English term and a semi-correct Norwegian translation term for 'machine translation' in the same document… Of course, there is no problem understanding that they are the same thing (from the context)…" T1.
"The overall meaning of the machine translated article is clear. There are few ambiguous sentences. Fluency is for the most part satisfactory. In some parts it is bad. There are short sections of text without any mistakes, but these are more the exception than the rule…" T3.
"There are several grammatical errors, but overall the meaning is clear." T4.
"Even though there are many mistakes in the grammar and the spelling, the meaning is often easy to understand…" T5.
RSS feed