Korean English Teachers' Decision Making Processes in Scoring Middle School Students' English Essays

Abstract

The primary goal of the present study is to examine Korean English teachers decision-making processes when scoring middle school students English essays. A total of nine Korean English teachers participated in this study and was asked to rate ten essays written on a scale of 10. During the scoring processes, they produced concurrent think-aloud protocols and the collected data was transcribed in full and then analyzed qualitatively. The analysis of the verbal reports revealed the following results. First, most of the raters were more oriented to judging grammatical accuracy than content when making scoring decisions. Second, the raters with longer teaching experiences were more likely to exhibit the characteristics of expert raters scoring behaviors, which were creating a scoring rubric, showing personal engagement with the writer, and displaying initial scanning of the whole essay. In this line of reasoning, it was assumed that the length of teaching experience could be one of the major factors which contribute to the individual differences in scoring processes. Also, based on the analysis, Korean English teachers 12 key scoring behaviors were formulated and these were categorized by three strategies, judgement, macro-judgment and interpretation strategies. Also, judgement and interpretation strategies were further divided into three focuses, language, organization, and content focuses. In the end, this study will shed light on variability in EFL raters cognitive processes in scoring essays as well as common strategies found in the variability. Moreover, this is expected to help rater trainers to develop fair scoring protocols and settle down more reliable writing assessment systems in Korean school environments.

본 연구는 한국인 영어 교사들이 중학교 학생들의 영어 작문 채점 시 보이는 의사결정 과정을 밝히고자 실시되었다. 9명의 한국인 중고등학교 영어 교사들이 10명의 학생들의 글을 총체적 방식으로 채점하였으며 동시에 사고구술을 진행하였다. 사고구술 결과를 분석한 결과 교사들은 공통적으로 문법성 판단에 가장 큰 비중을 들였으며, 글의 내용을 평가할 때는 구체적인 묘사를 가장 중요시하는 것을 알 수 있었다. 또한, 경력이 높은 교사일수록 선행연구에서 밝혀진 전문가 평가자의 특성을 보일 가능성이 높다는 것이 드러났는데, 그것은 글쓴이와의 교감, 평가표를 만드는 것, 평가 시작 전 글 전체를 훑어보는 것이었다. 이 연구를 통해 한국인 영어 교사들이 글쓰기 채점 시 보이는 개별적 차이를 알 수 있었으며 앞으로 교사 연수자들이 신뢰도 있고 타당도 높은 채점 기준을 설정할 때 발판이 되는 시사점을 제공할 수 있을 것으로 기대한다.