The Interlocutor Proficiency Effect on Test-taker Performance in a Paired Oral Assessment

Abstract

Paired speaking tests have increasingly gained popularity in the field of language assessment for their authentic ways of assessing interactive competence. Despite the benefits associated with paired speaking tests, questions have been raised in terms of their fairness, reliability, and construct validity. Given that paired assessments involve the interaction of two or more test-takers, a major concern is the effect the interlocutor‟s proficiency might have on their partners during the interaction. In this regard, a relatively small body of previous literature has yielded contradicting results. This study empirically examined the effects of interlocutor proficiency on test-taker performance in paired speaking tests. It used a scheme similar to that of Iwashita (1998) and Davis (2009). Twenty-four Korean university students were assigned to two major proficiency groups (i.e., high and low) and were asked to take two separate paired tests under different conditions: once paired with a partner at a similar proficiency level and once with a partner at a different proficiency level. A mixed-design two-way analyses of variance (ANOVA) were conducted using the pairing conditions as independent variables. First, the analysis of variance examined the statistical significance of the score differences between the two paired test conditions through composite scores (i.e., aggregated scores of the five analytic criteria, namely

grammar

vocabulary

pronunciation

fluency

and discourse management). Second, the scores for each of the analytic criteria were examined separately, again using ANOVA. Finally, each gender group was also analyzed to investigate whether gender was a factor that moderated the interlocutor proficiency effect. The analysis of the results indicated that the interlocutor‟s proficiency had no significant effect on the composite scores of participants. In addition, the interlocutor‟s proficiency had no statistically significant effect on any of the analytic scores assigned for the rating criteria. Furthermore, even when each gender group was examined separately, the interlocutor‟s proficiency turned out to have no statistically significant effects on the composite and analytic scores of female as well as male participants. Overall, the interlocutor‟s proficiency had no statistically significant effect on test-taker performance in the paired speaking tests. The present study has provided evidence in support of the use of paired speaking tests as a tool to measure speaking ability in a valid, reliable, and fair manner.