Engagement With Automated Writing Feedback in Mandated vs. Voluntary Conditions

Shaun J. Manning; Sookyung Cho (조수경)

doi:10.18095/meeso.2019.20.4.18

Engagement With Automated Writing Feedback in Mandated vs. Voluntary Conditions

Modern English Education
Abbr : MEESO
2019, 20(4), pp.18-30
DOI : 10.18095/meeso.2019.20.4.18
Publisher : The Modern English Education Society
Research Area : Humanities > English Language and Literature > English Language Teaching
Received : September 15, 2019
Accepted : November 8, 2019
Published : November 30, 2019

Shaun J. Manning ¹, Sookyung Cho ²

¹Hankuk University of Foreign Studies
²한국외국어대학교

Accredited

ABSTRACT

This study investigates student engagement with Écree, a Natural Language Processing (NLP) based automated writing evaluation (AWE) that gives feedback on content. In particular, this study examines how mandating AWE use or not, affects students’ behavioral and emotional engagement with AWE feedback. Two classes—one in which Écree use was mandatory (EM) and another in which it was optional (EO)—were surveyed and four volunteers from each class were tracked. Overall, the two groups differed in their evaluations of how much they incorporated AWE feedback into revisions as well as in the number of uploads to Écree. Tracking the focal students’ written drafts, Écree feedback, and interview data revealed that students from EM tended to incorporate Écree’s feedback into their revisions more than EO students. Unexpectedly, EM students developed quite negative views toward Écree, whereas EO students often referred to its usefulness: its systematicity, speed, and role as a reader. This study implies that AWE can be utilized as a supplemental tool in higher education and that students may well need training so as to maximize its positive effects.

KEYWORDS

EFL Writing/ AWE/ content feedback/ error correction / student perceptions

Citation status

* References for papers published after 2023 are currently being built.

[journal] Armel, D. / 1996 / The effects of required and optional computer-based note taking on achievement and instruction completing time / Journal of Educational Computing Research 14(4) : 329~344

[confproc] Attali, Y. / 2004 / Exploring the feedback and revision features of Criterion / Paper presented at the National Council on Measurement in Education

[journal] Attali, Y. / 2006 / Automated essay scoring with e-rater v.2 / Journal of Technology, Learning, and Assessment 4(3) : 1~30

[other] Baron, D. / 1998 / When professors get A’s and the machines get F’s / Chronicle of Higher Education : A56~

[journal] Brandl, K. / 2012 / Effects of required and optional exchange tasks in online language learning environments / ReCALL 24(1) : 85~107

[confproc] Burstein, J. / 2004 / CriterionSM:Online essay evaluation: An application for automated evaluation of student essays / Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence / AAAI Press : 3~10

[web] CCCC Executive Committee / 2004 / CCCC position statement on teaching, learning, and assessing writing in digital environments / https://cccc.ncte.org/cccc/resources/positions/digitalenvironments

[web] CCCC Executive Committee / 2006 / Writing assessment:A position statement / http://www.ncte.org/cccc/resources/positions/writingassessment

[journal] Chen, C. -F. E. / 2008 / Beyond the design of automated writing evaluation : Pedagogical practices and perceived learning effectiveness in EFL writing classes / Language Learning & Technology 12(2) : 94~112

[journal] Cheville, J. / 2004 / Automated scoring technologies and the rising influence of error / The English Journal 93(4) : 47~52

[confproc] Elliot, S. M. / 2004 / The impact of MY Access!™ use on student writing performance:A technology overview and four studies / Paper presented at the annual meeting of the American Educational Research Association

[book] Ericsson, P. F. / 2006 / Machine scoring of human essays: Truth and consequences / Utah State University Press

[journal] Farrell, T. S. C. / 2019 / Reflective practice framework for TESOL teachers : One teacher’s reflective journey / Reflective Practice 20(1) : 1~12

[journal] Foltz, P. W. / 1999 / Automated essay scoring: Applications to educational technology [Electronic Version] / Interactive Multimedia Electronic Journal of Computer-Enhanced Learning

[journal] Garland, K. / 2004 / The effects of mandatory and optional use on students’ ratings of a computer-based learning package / British Journal of Educational Technology 35(3) : 263~273

[journal] Grimes, D. / 2010 / Utility in a fallible tool: A multi-site case study of automated writing evaluation / Journal of Technology, Learning, and Assessment 8(6) : 4~44

[web] Heit, J. / 2018 / Écree / https://www.ecree.com

[journal] Stuart Jones / 2019 / Digital Technology in the 4IR and the Future of English Learning From the Perspective of Korean EFL University Students / 현대영어교육 / 현대영어교육학회 20(1) : 53~70

[journal] Lai, Y. H. / 2010 / Which do students prefer to evaluate their essays: Peers or computer program? / British Journal of Educational Technology 41(3) : 432~454

[book] Leki, I. / 2006 / Feedback in second language writing: Contexts and issues / Cambridge University Press : 266~286

[journal] Li, J. / 2015 / Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction / Journal of Second Language Writing 27(1) : 1~18

[book] Myers, M. / 2003 / Automated essay scoring: A cross-disciplinary approach / Lawrence Erlbaum Associates : 3~20

[journal] 박준희 / 2019 / An AI-based English Grammar Checker vs. Human Raters in Evaluating EFL Learners’ Writing / 멀티미디어 언어교육 / 한국멀티미디어언어교육학회 22(1) : 112~131

[report] Papageorgiou, S. / 2015 / The association between TOEFL iBT® test scores and the Common European Framework of Reference (CEFR) levels / Educational Testing Service

[book] Shermis, M. D. / 2003 / Automated essay scoring: A cross-disciplinary perspective / Lawrence Erlbaum Associates

[confproc] Shermis, M. D. / 2004 / The impact of automated essay scoring on high stakes writing assessments / Paper presented at the annual meeting of the National Council on Measurement in Education

[confproc] Shermis, M. D. / 2008 / The impact of automated essay scoring on writing outcomes / Paper presented at the Annual Meeting of the National Council on Measurement in Education

[web] Vantage Learning / 2007 / MY Access!® efficacy report / http://www.vantagelearning.com/learning-center/research/#MyAccess

[journal] Wang, Y. -J. / 2013 / Exploring the impact of using automated writing evaluation in English as a foreign language university students’writing / Computer Assisted Language Learning 26(3) : 234~257

[journal] Warschauer, M. / 2008 / Automated writing assessment in the classroom / Pedagogies : An International Journal 3(1) : 22~36

[journal] Warschauer, M. / 2006 / Automated writing evaluation : Defining the classroom research agenda / Language Teaching Research 10(2) : 1~24

[journal] Williamson, M. M. / 2004 / Validity of automated scoring : Prologue for a continuing discussion of machine scoring student writing / Journal of Writing Assessment 1(2) : 85~104

[journal] Zhang, Z. / 2017 / Student engagement with computer-generated feedback : A case study / ELT Journal 71(3) : 317~328

KJCKorea
Journal Central

Modern English Education 2023 KCI Impact Factor : 1.55

Engagement With Automated Writing Feedback in Mandated vs. Voluntary Conditions

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Modern English Education 2023 KCI Impact Factor : 1.55

Engagement With Automated Writing Feedback in Mandated vs. Voluntary Conditions

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (2)

REFERENCES (33) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.