Automated Essay Assessment Using Latent Semantic Analysis in Digital Grading Applications

The problem of manual essay scoring on a large scale has been identified as the “structural throughput constraint” in large-scale human writing assessment since Page's original 1966 proposal: a mean of 4.8 minutes per essay for a well-qualified human rater does not scale to a school district's writing curriculum, which typically produces tens of thousands of essays per grading period. Automated essay scoring systems have been based on a distributional semantics method called Latent Semantic Analysis (LSA) which uses truncated Singular Value Decomposition (SVD) to extract latent semantic structure from a term-document matrix that is independent of the exact lexical overlap between term and document. The statistical basis of automated essay scoring systems is Latent Semantic Analysis (LSA), a technique from distributional semantics, which was developed in the late 1990s by Landauer and colleagues using truncated Singular Value Decomposition (SVD) on a term-document matrix to obtain a set of latent semantic structure independent of exact lexical overlap. In this paper, 14,400 essays in three grade levels (4,840 Grade 8; 5,680 Grade 10; 3,880 Grade 12) and four prompt types (Persuasive: 3,840; Narrative: 3,200; Expository: 4,160; Argumentative: 3,200) were double-scored by certified raters for automated holistic essay scoring on a 6-point holistic rubric, with the inter-rater Quadratic Weighted Kappa (QWK) ranging from 0.84 to 0.92. A term-document matrix of 8,400 terms by 11,520 training essays (80% split; held-out test set: 2,880 essays) is TF-IDF weighted and reduced via truncated SVD to k = 300 latent dimensions, explaining 68.4% of total matrix variance; essay scores are predicted by ridge regression on the 300-dimensional LSA feature vector. LSA achieves QWK = 0.784 on the held-out test set-a 49.6% improvement over a word-count regression baseline QWK = 0.524) and a 14.6% improvement over direct TF-IDF cosine similarity QWK = 0.684)-reaching 93.3% of human-human inter-rater agreement (0.784/0.84) at zero marginal computational cost and 0.84 ms inference latency per essay, compared with a fine-tuned BERT scoring baseline that achieves QWK = 0.824 (98.1% of human agreement) at USD 0.008 per essay and 4.84 ms latency. Deployed across three school districts as a first-pass scoring assistant with human review triggered for 18.4% of essays flagged on confidence or rubric-boundary criteria, the system reduces mean grading time from 4.8 to 1.84 minutes per essay (61.7% reduction) and grading turnaround from 6.8 to 1.2 days (82.4% reduction).

Hali tarjima qilinmagan