Characteristics of EFL Writing Raters' Bias Patterns

Abstract

This study investigated the characteristics of the bias patterns of EFL writing raters according to their variables. In particular, the present study is devoted to finding out systematic bias patterns and their sources. For this purpose, the study examined 40 EFL raters results in holistic and analytic rating. The studys findings are summarized as follows. First, there were significant bias patterns in holistic and analytic rating. While bias patterns were explained more distinctively according to rater variables in holistic rating, it was hard to explain these bias patterns systematically according to rater variables in analytic rating. Second, the most frequent bias patterns were found in the EFL rater-writer interactions. This implies that raters have a difficulty in rating different levels of students at the same time: high, intermediate and low levels. Third, beliefs or values on writing instruction and assessment which are deeply lodged in much experienced EFL teachers interfered with rating, that is, raters tended to assess severely or leniently. On the other hand, inexperienced EFL teachers also demonstrated specific bias patterns in holistic rating due to their unfamiliarity with direct essay rating. Pedagogical implications of the study are that effective rater training and fine-tuned rating rubrics are needed to aid raters in reducing systematic bias patterns in EFL writing assessment. EFL raters have considerable problems in adjusting their score range, especially in assessing different levels of writing simultaneously. Consequently, more intensive training for rating various levels of writing will be the best measure to minimize EFL rater bias. Intensive and structured rating training is required for both experienced and inexperienced raters.