알고리즘 개발 및 타당도 검증

Abstract

In the era of information and communication technology, where computerized adaptive language testing continues to draw a keen attention from language testers, the present research attempts to develop a reliable and efficient algorithm of computerized adaptive language testing (CALT) and to investigate the extent to which the algorithm results in reliable and valid outcome. Employing an eclectic approach to analyzing real data as well as simulation data, the research has developed a highly reliable and valid algorithm on the basis of fundamental considerations including the difficulty level of initial items, the estimation method of the ability parameter, the item selection, the test length, the measurement error, the context effect, the item-based response time, etc. Focusing on the concurrent validity, the overall research findings suggest that the CALT version of the TEPS (Test of English Proficiency developed by SNU) accommodating the item-based response time factor is superior to the paper-based language test (PBLT) version of the TEPS in terms of measurement accuracy of overall English proficiency. Furthermore, the correlational analyses based on test-takers performance of CALT, PBLT, and TOP (the Test of Oral Proficiency) strongly support the plausibility of the test methods of the TEPS in which the grammar test is administered in a speeded manner and the listening comprehension test input is presented in a purely oral mode.