Different Warmup and Annealing Strategies for ANN Models to Predict Air Quality Index
Annotatsiya
Predicting outcomes based on tabular data is a common task in machine learning. Models like Multilayer Perceptron (MLP), Tabular Neural Network (TabNet), and Deep Factorization Machine (DeepFM) have been successful for such tasks. However, improving the training process can enhance model performance and convergence. One way to achieve this is by using learning rate schedules, such as warmups and annealing. These techniques help stabilize training in the early stages and refine model convergence toward optimal solutions. In this article, we shall explore how to use different warmup and annealing strategies to optimize MLP, TabNet, and DeepFM models for predicting Air Quality Index (AQI). The dataset including air quality index, pollutants concentrations effecting air pollution, weather data with 15 features and 164,000 samples are collected from 10 station in Tashkent, Uzbekistan. The results indicate that models using warmup and annealing learning rates outperform those that do not.