Modern English Education 2023 KCI Impact Factor : 1.55
-
pISSN : 1598-0782 / eISSN : 2586-6141
- https://journal.kci.go.kr/meeso
KJCKorea
Journal Central
Validity Argument for Post-Entry Oral Performance Assessment for International Students in Korean Higher Education
- Modern English Education
- Abbr : MEESO
- 2023, 24(), pp.91-109
- DOI : 10.18095/meeso.2023.24.1.91
- Publisher : The Modern English Education Society
- Research Area : Humanities > English Language and Literature > English Language Teaching
- Received : April 28, 2023
- Accepted : June 1, 2023
- Published : July 31, 2023
Han Jiyoung 1, Lee Kilryoung 1
1Hankuk University of Foreign Studies
Accredited
ABSTRACT
With an increasing number of international students in Korean universities, communication breakdowns among international students, domestic students, and faculty members have been an important issue. However, their language ability and difficulties have received little attention. Since English is a lingua franca, this study evaluated their English oral communication ability by designing a post-entry oral performance assessment (PEOPA), conducting a pilot test, and investigating the validity of PEOPA based on Toulmin’s (2003) argument structure. Firstly, a survey was conducted with 15 international students studying at a Korean university to ask about their English use and identify the target language use (TLU) domain. Among them, 11 took pilot PEOPA. A many-Facet Rasch analysis indicated that PEOPA tasks stably assessed the oral communication ability of participants and reliably separated them into different ability levels. Raters consistently assessed performances according to a scoring rubric reflecting intended construction, with each scoring category measuring distinct aspects of oral communication ability. Along with Rasch analysis, qualitative analysis of participants’ transcribed performances provided further backing for test validity. Ensured validity of the assessment allowed us to propose the implementation of PEOPA in universities in similar contexts. Finally, the implications of findings for the diagnosis phase of PEOPA are discussed.
Statistics
-
116 Viewed
-
78 Downloaded
Tools
Issue List
Citation status
* References for papers published after 2023 are currently being built.
[journal]
Altbach, P. G.
/ 2007
/ The internationalization of higher education: Motivations and realities
/ Journal of Studies in International Education
11(3-4)
: 290~305
[book]
Bachman, L. F
/ 2004
/ Statistical analyses for language assessment
/ Cambridge University Press
[journal]
Bachman, L. F.
/ 2005
/ Building and supporting a case for test use
/ Language Assessment Quarterly
2
: 1~34
[book]
Bond, T. G.
/ 2015
/ Applying the Rasch model: Fundamental measurement in the human sciences
/ Routledge
[journal]
Bonk, W. J.
/ 2003
/ A many-facet Rasch analysis of the second language group oral discussion task
/ Language Testing
20
: 89~110
[journal]
Brown, L.
/ 2008
/ The incidence of study-related stress in international students in the initial stage of the international sojourn
/ Journal of Studies in International Education
12(1)
: 5~28
[journal]
Campbell, J.
/ 2008
/ Asian students' voices : An empirical study of Asian students' learning experiences at a New Zealand university
/ Journal of Studies in International Education
12(4)
: 375~396
[journal]
Chang, S.
/ 2021
/ English medium instruction, English-enhanced instruction, or English without instruction : The affordances and constraints of linguistically responsive practices in the higher education classroom
/ TESOL Quarterly
55(4)
: 1114~1135
[journal]
최현실
/ 2018
/
A Study on the Adaptation of Foreign Students to Korean Universities
/ 인문사회과학연구
/ 인문사회과학연구소
(61)
: 71~94
[journal]
Crowther, D.
/ 2016
/ Linguistic dimensions of second language accent and comprehensibility : Nonnative listeners’ perspectives
/ Journal of Second Language Pronunciation
2(2)
: 160~182
[journal]
Csizmazia, R.
/ 2019
/ Challenges and internationalization of higher education in South Korea
/ International Journal of Humanities and Social Science
6(6)
: 11~18
[journal]
Doe, C.
/ 2014
/ Diagnostic English language needs assessment(DELNA)
/ Language Testing
31(4)
: 537~543
[journal]
D'Silva, F.
/ 2021
/ Developing disciplinary discourse in a first-year engineering course : The DELNA initiative
/ Discourse and Writing/Rédactologie
31
: 126~135
[book]
Eckes, T.
/ 2015
/ Introduction to many-facet Rasch measurement: Analyzing and evaluating rater- mediated assessments
/ Peter Lang
[journal]
Jon, Jae-Eun.
/ 2012
/ Power dynamics with international students : From the perspective of domestic students in Korean higher education
/ Higher Education
64(4)
: 441~454
[journal]
Jon, Jae-Eun.
/ 2013
/ Realizing internationalization at home in Korean higher education
/ Journal of Studies in International Education
17(4)
: 455~470
[journal]
Jon, Jae-Eun.
/ 2014
/ The emergence of a regional hub : Comparing international student choices and experiences in South Korea
/ Higher Education
67(5)
: 691~710
[journal]
Kane, M.
/ 2012
/ Validating score interpretations and uses
/ Language Testing
29(1)
: 3~17
[journal]
Kane, M.
/ 2013
/ Validating the interpretations and uses of test scores
/ Journal of Educational Measurement
50
: 1~73
[book]
Kane, M.
/ 2021
/ The Routledge handbook of language testing
/ Routledge
: 32~47
[journal]
Kasper, G.
/ 1991
/ Research methods in interlanguage pragmatics
/ Studies in Second Language Acquisition
13(2)
: 215~247
[thesis]
Kim, Hyekyeng
/ 2015
/ The effects of pragmatic instruction on Korean university students with different language proficiency levels
/ doctor
/ Hankuk University of Foreign Studies
[journal]
Kim, Hyun Jung.
/ 2015
/ A qualitative analysis of rater behavior on an L2 speaking assessment
/ Language Assessment Quarterly
12
: 239~261
[web]
Language Requirements
/ 2022
/ Hankuk University of Foreign Studies
/ https://sites.google.com/view/internationalhufs/English-Admission-Guide/languagerequirements
[journal]
이지우
/ 2014
/
An investigation into native English-speaking and Korean raters’ judgments of Korean English learners’ pronunciations.
/ 현대영어교육
/ 현대영어교육학회
15(1)
: 195~216
[journal]
Lee, Yong-won.
/ 2015
/ Future of diagnostic language assessment
/ Language Testing
32(3)
: 295~298
[journal]
Lim, G. S.
/ 2011
/ The development and maintenance of rating quality in performance writing assessment : A longitudinal study of new and experienced raters
/ Language Testing
28
: 543~560
[journal]
Lunz, M. E.
/ 1990
/ Measuring the impact of judge severity on examination scores
/ Applied Measurement in Education
3(4)
: 331~345
[journal]
May, L.
/ 2011
/ Interactional competence in a paired speaking test : Features salient to raters
/ Language Assessment Quarterly
8(2)
: 127~145
[journal]
May, L.
/ 2020
/ Developing tools for learning oriented assessment of interactional competence : Bridging theory and practice
/ Language Testing
37(2)
: 165~188
[web]
Ministry of Education
/ 2022
/ Statistics of foreign students in domestic higher education institutions in 2021
/ https://www.moe.go.kr/sn3hcv/doc.html?fn=48fb0857a562dd3141f0b6062d1fdba8&rs=/upload/synap/202202/
[journal]
Muller, A.
/ 2011
/ Addressing the English language needs of international nursing students
/ Journal of Academic Language and Learning
5(2)
: A14~A22
[report]
Myford, C. M.
/ 2000
/ Monitoring sources of variability within the Test of Spoken English assessment system
/ Educational Testing Service
[journal]
Ockey, G. J.
/ 2021
/ Human versus computer partner in the paired oral discussion test
/ Applied Linguistics
42(5)
: 924~944
[book]
Paul, T. H.
/ 2007
/ Doing conversation analysis: A practical guide
/ Sage
[journal]
박순영
/ 2016
/
Foreign Students’Adaptation to College Life in Korea - Focusing on the Social Relations -
/ 지역사회연구
/ 한국지역사회학회
24(2)
: 75~102
[journal]
Poyrazli, S.
/ 2006
/ Marital status, ethnicity, academic achievement, and adjustment strains
/ College Student Journal
40(4)
: 767~780
[journal]
Read, J.
/ 2008
/ Identifying language needs through diagnostic assessment
/ Journal of English for Academic Purposes
7(3)
: 180~190
[journal]
Read, J.
/ 2015
/ Issues in post-entry language assessment in English-medium universities
/ Language Teaching
48(2)
: 217~234
[journal]
Roberts, P. A.
/ 2018
/ Towards a reframing of student support : A case study approach
/ Higher Education
75(1)
: 19~33
[journal]
Roever, C.
/ 2022
/ What scores from monologic speaking tests can(not)tell us about interactional competence
/ Language Testing
39(1)
: 7~29
[journal]
Roever, C.
/ 2018
/ Speaking in turns and sequences : Interactional competence as a target construct in testing speaking
/ Language Testing
35(3)
: 331~355
[journal]
Saito, K.
/ 2020
/ Multi-or single-word units? The role of collocation use in comprehensible and contextually appropriate second language speech
/ Language Learning
70(2)
: 548~588
[book]
Taguchi, N
/ 2018
/ Critical reflections on data in second language acquisition
/ John Benjamin
: 7~32
[thesis]
Thibault, N
/ 2022
/ Student-faculty interaction experiences of international students in Korea: A phenomenology study
/ doctor
/ The American College
[book]
Toulmin, S. E.
/ 2003
/ The uses of argument
/ Cambridge University Press
[journal]
Youn, Soo Jung.
/ 2015
/ Validity argument for assessing L2 pragmatics in interaction using mixed methods
/ Language Testing
32(2)
: 199~225
[journal]
Yu, B.
/ 2013
/ Asian international students at an Australian university : Mapping the paths between integrative motivation, competence in L2 communication, cross-cultural adaptation and persistence with structural equation modeling
/ Journal of Multilingual and Multicultural Development
34
: 727~742
[journal]
Yu, B.
/ 2019
/ Language and interpersonal resource predictors of psychological and sociocultural adaptation : International students in Hong Kong
/ Journal of Studies in International Education
23(5)
: 572~588