LoRA Effectiveness in Text Generalization Tasks: Comparison of T5 and uzT5 Models
Annotatsiya
This paper presents an analysis of the application of the Low-Rank Adaptation method for the task of monolingual text generation in Uzbek. We used the T5-base, T5-Large and uzT5 models to determine which one shows the best results when using LoRA, and also compared their performance with traditional fine tuning. A dataset consisting of text from 5,000 news articles on the Kun.uz, Daryo.uz, Xabar.uz and Qalampir.uz platforms was utilized, with 4,000 of these articles designated for training purposes and the remaining 1,000 reserved for testing. The performance of the models was evaluated using the metrics BLEU, ROUGE-1, ROUGE-2, ROUGE-L and ROUGE-LSUM. Our results showed that the uzT5-base model with Low-Rank Adaptation method r=256 and α=512 parameters demonstrates the highest performance among all the considered models, providing the best values of the ROUGE and BLEU metrics with a moderate number of training parameters, which makes it more computationally efficient compared to mT5-Large.