Course Feedback Summarization Using Longformer Transformer Model in Linguistic Education Feedback Systems
Annotatsiya
Language-learning platforms collect open-text end-of-course feedback at a volume that exceeds what instructors or administrators can read comment-by-comment: a single course's concatenated feedback corpus in this study averages 4,704 tokens, 9.2 times the 512-token input limit of standard BERT-based summarisers, forcing truncation-based methods to discard the majority of submitted feedback before any summarisation occurs. Longformer (Beltagy, Peters, and Cohan, 2020) replaces full quadratic self-attention with a linear-complexity combination of sliding-window local attention and a small number of global-attention tokens, extending practical input length to 4,096 tokens and enabling native, untruncated summarisation of most course feedback corpora in this domain. This paper applies Longformer to 1,840 language-course feedback corpora (77,280 total comments; 6,491,520 words; six languages) on a multilingual language-learning platform, benchmarking it against LSA (extractive), BERT-Extractive (truncated to 512 tokens), and BART (abstractive, 1024-token chunked) baselines. Longformer achieves ROUGE$-\mathbf{1}=\mathbf{0. 4 6 8}$, exceeding BART by 10.4% and LSA by 44.4%, and-reflecting its core architectural advantage-achieves 86.4% topic coverage of the full feedback corpus against BERT-Extractive's 24.8% (a 248.4% relative improvement), at a factual-consistency score of 0.844, intermediate between the inherently safer extractive baselines (0.884-0.904) and the more hallucination-prone chunked BART approach (0.764). Deployed for one academic term, Longformer summaries reduced mean instructor feedback-review time from 42 to 12 minutes per course (−71.4%) and increased the proportion of instructors who identified at least one actionable item from 38.4% to 58.4% (+52.1%), while aggregated cross-course summaries supported 12 curriculum adjustments that manual review of 77,280 individual comments would not practically have surfaced. A targeted audit found that among 248 courses exhibiting a majority-positive-with-vocal-minority-concern feedback pattern, only 34.7% of generated summaries explicitly retained the minority concern; a separate manual review found 7.0% of sampled summaries contained at least one unsupported (hallucinated) claim; and summary quality degraded by 33.3% (ROUGE-1) for courses with substantial target-language (non-English) feedback, reflecting the model's English-centric pretraining [4].
Hali tarjima qilinmagan