AI-Powered Speech Fluency Assessment in Real-Time Language Learning Applications

Automated speech fluency assessment enables language-learning applications to deliver immediate, scalable feedback that human tutors cannot match in volume-but only if the automated scores align closely enough with trained human raters to be trusted, arrive quickly enough not to disrupt conversational pacing, and diagnose the specific fluency dimensions a learner needs to address rather than returning a single opaque composite score. This paper fine-tunes Wav2Vec2, Facebook's self-supervised speech representation model, on a multi-dimensional fluency annotation task covering four dimensions (pronunciation accuracy, speaking rate, pause ratio, and prosody) across 691,200 scored utterances from 2,400 English language learners spanning three CEFR bands (A1-A2: 720 learners, 30.0%; B1-B2: 1,080, 45.0%; C1-C2: 600, 25.0%) and 28,800 monitored sessions. Against held-out human-rater scores, Wav2Vec2 achieves Pearson r = 0.864 and RMSE = 7.4 (0-100 scale), outperforming a rule-based threshold system$(\mathbf{r}=\mathbf{0. 5 2 4}; +\mathbf{6 4. 9 {\%}})$, an SVM on acoustic features$(\mathbf{r}=\mathbf{0. 6 8 4}$; +26.3%), and an LSTM sequence model$(\mathbf{r}=\mathbf{0. 7 6 4}$; +13.1%), while meeting the 300 ms real-time latency target at 184 ms mean inference time. Sub-dimension correlation is strongest for speaking rate$(\mathbf{r}=\mathbf{0. 9 2 4})$and weakest for prosody$(\mathbf{r}=\mathbf{0. 7 6 4})$, revealing where the model still falls short of human-level judgement. Deployed in a live language-learning application, learners receiving automated fluency feedback showed a 41.3% increase in voluntary practice-session engagement and a 58.3% increase in mean weekly practice time relative to a matched control group. L1 interference, accent-variant generalisation, low-bandwidth audio degradation, and a documented score-inflation vulnerability via deliberate speaking-rate manipulation each constrain the model's reliable scope, with concrete mitigations described for each.

Hali tarjima qilinmagan