An analysis of the linguistic elements of the text in the English reading section of the College Scholastic Ability Test. Modern English Education (대학수학능력시험 영어 읽기 지문의 언어적 요소 분석)

문지현; Haedong Kim (김해동)

An analysis of the linguistic elements of the text in the English reading section of the College Scholastic Ability Test. Modern English Education

Modern English Education
Abbr : MEESO
2017, 18(1), pp.193-211
Publisher : The Modern English Education Society
Research Area : Humanities > English Language and Literature > English Language Teaching
Published : February 28, 2017

문지현 ¹, Haedong Kim ¹

¹한국외국어대학교

Accredited

ABSTRACT

This study aims to investigate changes of the text-difficulty in the English reading section of the College Scholastic Ability Tests (CSATs) in Korea. Texts from the 1994 to 2016 CSATs, were analyzed by Coh-Metrix, a linguistic computational program. The results of the readability indices, obtained by Flesch Reading Ease and Flesch Kincaid Grade Level, indicated that the difficulty of the text in the CSATs has not been maintained. The following linguistic elements have increased: the mean number of syllables in words, the mean number of letters in words, the number of sentences, the mean number of words in each sentence, the syntactic complexity and the average number of sentences in each paragraph. Two previous CSATs, from 2001 and 2013, were experimented on 79 high school students. The results showed that the 2013 CSAT was more difficult than the 2001 CSAT. Based on the results from the analysis and the experiment, it was concluded that the text in the English reading section of CSAT has gradually become difficult. We suggest that linguistic analysis should be used as a tool for estimating the level of text-difficulty in the English reading section when professionals write test-items for the CSAT.

KEYWORDS

College Scholastic Ability Test/text difficulty/linguistic analysis

Citation status

* References for papers published after 2023 are currently being built.

[journal] Alderson, J. C. / 1985 / The effect of students’ academic discipline on their performance on ESP reading tests / Language Testing 2(2) : 192~204

[book] Brown, H. D. / 2010 / Language assessment: Principles and classroom practices / Longman

[journal] 장경숙 / 2004 / A model of predicting item difficulty of the reading test of College Scholastic Ability Test / Foreign Languages Education / 한국외국어교육학회 11(1) : 111~130

[book] Clapham, C. / 1996 / The development of IELTS: A study of the effect of background knowledge on reading comprehension / Cambridge University Press

[journal] Davey, B. / 1984 / The interaction of reader and task factors in the assessment of reading comprehension / The Journal of Experimental Education 52(4) : 199~206

[journal] Fincher-Kiefer, R. / 1992 / The role of prior knowledge in inferential processing / Journal of Research in Reading 15(1) : 12~27

[report] Freedle, R. / 1991 / The prediction of SAT reading comprehension item difficulty for expository prose passages for three item types: Main idea, inference and supporting idea items / Educational Testing Service

[report] Freedle, R. / 1993 / The prediction of TOEFL reading comprehension item difficulty for three item types: Main idea, inference and supporting idea items / Educational Testing Service

[journal] Gilhooly, K. J. / 1980 / Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1, 944 words / Behavior Research Methods & Instrumentation 12(4) : 395~427

[journal] Graesser, A. C. / 2004 / Coh-metrix:Analysis of text on cohesion and language / Behavior Research Methods, Instruments, & Computers 36(2) : 193~202

[journal] Guttman, L. / 1967 / Systematic construction of distractors for ability and achievement test items / Educational and Psychological Measurement 27(3) : 569~580

[journal] 전문기 / 2011 / A Corpus-based Analysis of the Continuity of the Reading Materials in Middle School English 1 and 2 Textbooks with Coh-Metrix. / 언어과학연구 / 언어과학회 (56) : 201~218

[journal] 전문기 / 2009 / A Corpus-based analysis of middle school English 1 textbooks with Coh-Metrix / 영어교육연구 / 팬코리아영어교육학회 21(4) : 265~292

[journal] 전문기 / 2010 / A corpus-based linguistic analysis of high school English textbooks / Foreign Languages Education / 한국외국어교육학회 17(1) : 209~233

[journal] 지슬기 / 2014 / A comparison of the effects of test-item type and text familiarity on results of an English reading test / Foreign Languages Education / 한국외국어교육학회 21(1) : 215~239

[journal] Jin, Kyung-Ae / 2004 / The Prediction of English Item Difficulty in College Scholatic Ability Test / 영어교육 / 한국영어교육학회 59(1) : 267~278

[journal] Kim, Choong-Bae / 1994 / Chulje gollandoui yecheukgwa silje [Prediction and reality of item-writing difficulty] / Goryeodaehakgyo Sadaenonjip [Journal of College of Education in Korea University] 19 : 115~136

[journal] 김해동 / 2013 / An analysis of journal articles on ELT materials in Korea / 현대영어교육 / 현대영어교육학회 14(1) : 225~242

[journal] 김정렬 / 2012 / An analysis of the continuity of elementary and middle school English textbooks using Coh-Metrix. / 영어교육 / 한국영어교육학회 67(2) : 319~341

[confproc] Kim, Jun-Shik / 2016 / What matters in item difficulty for the criterion-referenced testing of English in the CSAT / Paper presented at the 2016 KELTA Conference

[journal] 김낙복 / 2008 / A corpus-based lexical analysis of the foreign language (English) test for the college scholastic ability test (CSAT) / 영어어문교육 / 한국영어어문교육학회 14(4) : 201~221

[journal] 김용석 / 2003 / The construct validity of multiple choice items and the prediction of item difficulty for TOEIC reading comprehension / 영어교육연구 / 팬코리아영어교육학회 15(4) : 191~214

[journal] 권오량 / 2015 / A history of policies regarding the English section of Korea’s College Scholastic Ability Test / 영어교육 / 한국영어교육학회 70 : 3~34

[journal] 권서경 / 2014 / The effects of the EBS books-CSAT linkage policy on vocabulary difficulty of the English section in the CSAT / 영어교과교육 / 한국영어교과교육학회 13(4) : 97~121

[report] Lee, Byungmin. / 2014 / Introduction of criterion referenced test in CSAT English / Korean Educational Development Institute

[journal] Levine, A. / 1994 / The four-dimensional model : Interaction of schemata in the process of EFL reading comprehension / TESL Canada Journal 11(2) : 71~84

[book] McNamara, D. S. / 2014 / Automated evaluation of text and discourse with Coh-Metrix / Cambridge University Press

[web] Ministry of Education / 2015 / Plan for 2018 CSAT / http://www.moe.go.kr/web/100026/ko/board/view.do?bbsId=294&pageSize=10&currentPage=0&encodeYn=N&boardSeq=60845&mode=view

[journal] Park, Gi-Pyo / 2004 / Comparison of L2 listening and reading comprehension by university students learning English in Korea / Foreign Language Annals 37(3) : 448~458

[book] Qian, D. / 2002 / Validation the role of depth of vocabulary knowledge in assessing reading for basic comprehension tasks in TOEFL 2000 / Educational Testing Service

[confproc] Rowe, M. / 2006 / An analysis of a standardized reading ability test: What do questions actually measure? / Proceedings of the Seventh International Conference of the Learning Sciences : 627~633

[journal] Sadoski, M. / 1993 / Impact of concreteness on comprehensibility, interest, and memory for text : Implications for dual coding theory and text design / Journal of Educational Psychology 85(2) : 291~304

[report] Scheuneman, J. / 1991 / Effects of prose complexity on achievement test item difficulty / Educational Testing Service

[thesis] Sung, Yoon-Mi. / 2003 / Factor analysis of English test scores in the College Scholastic Ability Test and implications / doctor / Inha University

KJCKorea
Journal Central

Modern English Education 2023 KCI Impact Factor : 1.55