COMPARATIVE ANALYSIS OF TRANSFORMER-BASED FEATURE EXTRACTION AND CLASSIFIER PERFORMANCE: AN EMPIRICAL STUDY ON IoT-ENABLED INDUSTRIAL SYSTEMS

This paper presents a comprehensive comparative analysis of five feature extraction methods (raw features, PCA, deep autoencoder, variational autoencoder, and Transformer) combined with six classification algorithms (logistic regression, SVM, MLP, XGBoost, LightGBM, CatBoost) for industrial IoT systems. Experiments conducted on three publicly available datasets-IoT predictive maintenance, smart logistics, and smart manufacturing-demonstrate that raw features combined with gradient boosting classifiers achieved the highest classification accuracy on logistics data, matching or exceeding the performance of sophisticated Transformer-based feature extraction paired with neural classifiers. Through rigorous five-fold stratified cross-validation and comprehensive performance analysis, our results confirm that feature extraction effectiveness is highly dataset-dependent, and when domain expertise produces quality features, additional deep transformation yields minimal benefit while increasing computational costs. We provide evidence-based guidelines for practitioners to select appropriate feature extraction methods based on data characteristics, performance requirements, and computational constraints. These findings have significant implications for resource-constrained IoT deployments where computational efficiency is paramount.

Hali tarjima qilinmagan