Construct Validation Study on SNUCREPT (Seoul National University Criterion-Referenced English Proficiency Test)

Abstract

The first pilot test for Seoul National University Criterion-Referenced English Proficiency Test (SNUCREPT) was developed to conduct a validation study to set a framework on which to develop the standardized EFL test battery, which will be used to measure English proficiency in a criterion-referenced manner. Designed to be a speed test, the total test of 250 items lasted approximately 150 minutes. Two analytic approaches, quantitative and qualitative approaches were employed to conduct the content analyses. Approximately 1,000 college students participated in this study. The descriptive statistics show that the distribution approximated normal distribution. The test indices (reliability, facility, discriminability) meet the requirement of a desirable CR test. The test methods were analyzed qualitatively and quantitatively to investigate the extent of their authenticity. Some of the findings help identify the problematic formats of widely used EFL test batteries, thus providing invaluable information for improving the formats of SNUCREPT. Application of Item Response Theory (lRT) was validated through checking the dimensionality assumption with Stout's factor analysis methods. IRT allows us to estimate more precise measurement of ability and item indices than does classical testing theory. Correlational analyses were done to validate the SNUCREPT in relation to Test of Spoken English (TSE). The results show a high correlation between TSE and SNUCREPT, which supports the rationale of language tests being speed tests. The first pilot test results lay a systematic basis on which to revise the format of SNUCREPT and to develop the second pilot test, which is to be further validated to finalize the testing specifications of SNUCREPT.