An Application of the Rasch Model to Assessing an English Achievement Test in a High School Setting

Shim Jae-woo (심재우); Keiko Samimy

An Application of the Rasch Model to Assessing an English Achievement Test in a High School Setting

Modern English Education
Abbr : MEESO
2010, 11(2), pp.278-294
Publisher : The Modern English Education Society
Research Area : Humanities > English Language and Literature > English Language Teaching

Shim Jae-woo ¹, Keiko Samimy ²

¹전북대학교
²The Ohio State University

Accredited

ABSTRACT

This study investigated the acceptability of reading test items developed by non-native Korean teachers of English in a Korean high school by the application of the Rasch model analysis. Data on 17 multiple-choice items of reading achievement for 256 high school students at a high school were collected. The data were submitted to Winstep, a Rasch analysis software, to study general model fit, item difficulty measures, person ability measures, fit indices, reliability, and separation indices. The results of the first Rasch analysis of 17 items indicated 5 items did not fit the Rasch model. The second analysis of the 12 items, after the deletion of 5 misfit items, revealed that the set of test items did not match individual persons’ ability or were poorly targeted with high-and low- ability persons. The content analysis of those 5 misfit items confirmed that the teachers had made serious mistakes in construction. Differential item functioning showed that only 1 item of the 12 test items was biased against gender. The results of the study suggested that Korean teachers of English need more thorough preparation and a higher sense of responsibility for constructing items and assigning grades to their students.

KEYWORDS

reading test items/classroom assessment/Rasch model analysis

Citation status

* References for papers published after 2023 are currently being built.

[book] Bachman, L. F. / 1996 / Language testing in practice / Oxford University Press

[book] Bond, T. / 2007 / Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.) / Lawrence Erlbaum

[book] Brown, D. / 2004 / Language assessment: principles and classroom practices / Person Education

[book] Brown, A. / 2007 / Interviewer style and candidate performance in the IELTS oral interview. in : Studies in language testing (19) IELTS collected papers / Cambridge University Press : 37~60

[book] Grabe, W. / 2000 / Reading research and its implications for reading assessment. in : Studies in language testing (9): Fairness and validation in language assessment / Cambridge University Press : 226~262

[journal] Hillocks, G. / 1984 / A taxonomy of skills in reading and interpreting fiction / American Educational Research Journal 21(1) : 7~24

[book] Hughes, A. / 2003 / Testing for language teachers / Cambridge University Press

[journal] Kulikowich, J. / 2008 / Evaluating fifth-and sixth-grade students’expository writing: Task development, scoring, and psychometric issues / Reading and Writing 21 : 153~175

[journal] Lee, C. / 1995 / Signifying as a scaffold for literary interpretation / Journal of Black Psychology 21(4) : 357~381

[book] Merryless, B. / 2007 / A survey of examiner attitudes and behavior in the IELTS oral interview. in : Studies in language testing (19) IELTS collected papers / Cambridge University Press : 142~178

[book] McNamara, T. / 1996 / Measuring second language performance / Addison Wesley Longman

[book] Moore, T. / 2007 / Authenticity in the IELTS academic module writing test: A comparative study of task 2 items and university assignments. in : Studies in language testing (19) IELTS collected papers / Cambridge University Press : 197~248

[journal] Waugh, R. / 1999 / Approaches to studying for students in higher education: A Rasch measurement model analysis / British Journal of Educational Psychology 69 : 63~79

KJCKorea
Journal Central

Modern English Education 2023 KCI Impact Factor : 1.55