Efforts have been made to improve the output of automated essay evaluation systems to emulate human ratings in the field of applied linguistics. The current study aimed to explore the textual features that distinguish good from poor second-language (L2) writing using Coh-Metrix version 3.0 and to compare the output of an online text analysis tool to human rating. Sixty essays were collected from tertiary-level on-campus essay writing contests in 2017 - 2018. Three experienced English instructors rated the essays using a rubric from Jacobs et al. (1981). The Coh-Metrix output, including indices of descriptive, text easability, lexical diversity, connectives, latent semantic analysis (LSA), word information, and L2 readability, were evaluated by correlation analyses and independent t-tests. The results showed that higher scorers wrote longer, with a greater number of sentences and paragraphs. They were also likely to use more concrete words, more frequent words, and better cohesion, all of which contributed to readability. The pedagogical implications and limitations of using Coh-Metrix for L2 writing testing and instruction were further discussed.