Assessing Interactional Competence of Advanced-Level Korean EFL Students Through L2 Paired Discussion Tasks

Abstract

In recent years, interaction-based speaking tests, including paired and group oral tests, have been considered as viable formats of second language (L2) speaking assessment. A promising aspect of such tests is that they can provide authentic and real conversation-like tasks for test takers to participate in, enable us to investigate L2 learners interactional competence as one crucial dimension of L2 speaking proficiency, and thereby increase the validity of speaking scores obtained from such tests. This promising characteristics of paired speaking tests has led to a number of research studies exploring the nature of interactional competence both theoretically and empirically as administering actual interaction-based tests in varying formats and contexts. However, little research studies have been conducted on the characteristics of interactive tasks and raters including rating scales they use which highly affect the quality of test takers speech performance and their ratings as the core variables in the context of speaking assessments. In response to this need for research, this study aims to investigate the feasibility of using paired discussion tasks to assess interactional competence and an existing scoring rubric for the construct in the Korean EFL context through both quantitative and qualitative methodologies. For this study, two paired discussion tasks were prepared and administered to 40 Korean EFL students at an advanced level of English proficiency. A total of 20 pairs of interlocutors were created and each pair completed two of the same discussion tasks. The elicited samples were rated first by two native English speaking raters on the dimension of the interlocutors interactional competence and then by two EFL trained raters on their general speaking proficiency. The elicited speech samples were also analyzed in terms of the types of interactional features operationalized during the test tasks and how raters perceived the features for assessing interlocutors interaction. Along with the patterns of salient interaction-related features in the test takers spoken responses, the reliability and validity of the obtained speaking scores were also examined. In addition, to investigate the raters perception of the tasks and scoring rubrics used for this study, their comments were collected and analyzed. The results of the study suggest that the paired discussion tasks not only achieved the acceptable levels of inter rater reliability but also elicited a variety of interactional features from the interlocutors. First, a total of 18 interactional features were identified through the discussion tasks, each of which appeared with different frequencies in this study. This result has provided some validity evidence supporting the use of paired discussion tasks to elicit interaction-based speech performance. Second, the qualitative analysis of the interlocutors response based on the raters comments has demonstrated that in-depth investigation of interactional features in conjunction with task characteristics may be useful and effective. For example, considering the characteristics of discussion tasks, the raters focused on how the topics were appropriately developed during the interaction for the given tasks, enabling the raters to make sophisticated identification of interactional competence. In this regard, clear limitations of the existing rating scale for interactional competence were exposed since the descriptors of the used rating scale did not fully reflect the actual speech performance. In addition, raters feedback on the rating scale revealed that making consistent and valid judgments was difficult for them due to the limited levels of interaction patterns and task completion status. Some implications and future research directions seem to arise from the major findings of this study. First, the current study empirically examined not only the reliability and validity of scores from paired discussion tasks but also the feasibility of using such paired oral tasks to elicit abundant interactional features as evidence of the interlocutors interactional competence. This has also suggested the need for further research on exploring interactional resources in the context of varying forms and structures of tasks. Second, the study provided helpful suggestions for constructing rating scales for interactional competence. For teachers and assessors, it is recommended to develop a rating scale with several levels of criteria and detailed descriptors of each level based on empirical analysis. Lastly, further studies on raters perspectives on the conceptualization of interactional competence and elicited speech performance are needed to provide them with guidance for making consistent and valid judgments, thereby ensuring the fairness of administering paired discussion tasks in the context of L2 speaking assessment.