AUTOMATED CONSTRUCTION OF A PUNCTUATION-TAGGED CORPUS FOR TRANSFORMER–BILSTM-BASED PUNCTUATION RESTORATION IN UZBEK

This study proposes an automated algorithm for constructing a punctuation-annotated dataset for training Transformer–BiLSTM models in Uzbek. The process includes text normalization, punctuation-aware tokenization, token-level labeling, dataset partitioning, and conversion into CoNLL format. A total of 11 labels are used, and the resulting corpus contains 11,888,213 tokens. The proposed method is intended for preparing large-scale annotated Uzbek data for punctuation restoration.

Hali tarjima qilinmagan