@article{ART003367909},
author={Hwa-Jin Ahn},
title={A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince},
journal={The Journal of General Education},
issn={2465-7581},
year={2026},
number={36},
pages={159-194},
doi={10.24173/jge.2026.07.31.5}
TY - JOUR
AU - Hwa-Jin Ahn
TI - A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince
JO - The Journal of General Education
PY - 2026
VL - null
IS - 36
PB - Da Vinci Mirae Institute of General Education
SP - 159
EP - 194
SN - 2465-7581
AB - This study proposes an LLM-based methodology for evaluating French literature essay examinations and validates its effectiveness using 30 mid-term exam scripts evaluated by the instructor who both designed and graded the assessment. Unlike conventional validation studies employing independent external raters, this study examines the extent to which an LLM can reproduce the evaluation standards of that instructor.
Focusing on comparative essays on Camus’s L’étranger and Saint-Exupéry’s Le Petit Prince, we developed an LLM scoring pipeline (utilizing Claude Sonnet 4.6) comprising: (1) question-specific credit/penalty matrices, (2) Dual-Reference Fusion (DRF) merging analytical and summary model answers, and (3) multi-axis scoring combining textual fidelity and stance clarity, augmented by a Stance Coherence Check. Quantitative analysis revealed a strong correlation between the AI and the human grader (Pearson r=0.786, Spearman ρ=0.651). Qualitatively, while the LLM exhibited relative leniency toward lower-tier responses, the human instructor more actively recognized and rewarded the intellectual potential in top-tier essays. These findings demonstrate that a multi-stage prompt design aligned with pedagogical intent enables LLMs to stably track the nuances of humanistic reasoning. Ultimately, this study underscores the practical viability of positioning LLMs not as a replacement, but as a first-pass "co-rater" in humanities assessment. While this methodology may extend to other humanities essays involving dual-text comparative structures, the empirical validation reported here is limited to the two works and single course examined.
KW - large language model;automated critical essay scoring;French literature education;Dual-Reference Fusion (DRF);Stance Coherence Check;human –AI co-rating
DO - 10.24173/jge.2026.07.31.5
ER -
Hwa-Jin Ahn. (2026). A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince. The Journal of General Education, 36, 159-194.
Hwa-Jin Ahn. 2026, "A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince", The Journal of General Education, no.36, pp.159-194. Available from: doi:10.24173/jge.2026.07.31.5
Hwa-Jin Ahn "A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince" The Journal of General Education 36 pp.159-194 (2026) : 159.
Hwa-Jin Ahn. A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince. 2026; 36 : 159-194. Available from: doi:10.24173/jge.2026.07.31.5
Hwa-Jin Ahn. "A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince" The Journal of General Education no.36(2026) : 159-194.doi: 10.24173/jge.2026.07.31.5
Hwa-Jin Ahn. A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince. The Journal of General Education, 36, 159-194. doi: 10.24173/jge.2026.07.31.5
Hwa-Jin Ahn. A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince. The Journal of General Education. 2026; 36 159-194. doi: 10.24173/jge.2026.07.31.5
Hwa-Jin Ahn. A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince. 2026; 36 : 159-194. Available from: doi:10.24173/jge.2026.07.31.5
Hwa-Jin Ahn. "A Methodology for LLM-Based Assessment of Literature Essay Examinations : From the Examiner’s Perspective through a Case Study on Camus’s L’Étranger and Saint-Exupéry’s Le Petit Prince" The Journal of General Education no.36(2026) : 159-194.doi: 10.24173/jge.2026.07.31.5
