HYDRA-XAI dual-backbone disaster scene recognition using ResNet50-Swin transformer feature fusion, explainable evidence, and an operational recommender

The fast identification of scenes of disasters in diverse imagery is important for emergency response; yet, the operational streams are overwhelmingly constituted by non-disasters, having few life-threatening human-inflicted damage scenarios. In this paper, we introduce HYDRA-XAI, which is an end-to-end system with 6 disaster classifications based on global Transformer context and local convolutional evidence. Specifically, the Frozen ResNet50 and Swin Transformer Tiny architectures yield the 2048-dim and 768-dim representations, which are concatenated to create a 2816-dim representation. This is subsequently classified via the XGBoost and SVM-RBF classifiers, whose probabilities are averaged through mean pooling. Multimodal evidence to aid class-specific operation recommendation is generated using Grad-CAM++, Transformer attention rollout, and LIME Top-K superpixels. Duplicate pairs were grouped, augmentation limited to the training subset, and the entire model was evaluated against a leak-proof testing dataset with 3,561 images, where each class contains 508-926 images. The trained ensemble model obtained the following metrics: Accuracy = 0.9935, Macro-F1 = 0.9929, Weighted-F1 = 0.9935, and macro one-vs-rest ROC-AUC = 0.9999. The train/test gap of Macro-F1 is 0.0069, with bootstrap 95% confidence intervals at [0.9898, 0.9957]. Escalation precision was boosted to 0.9972 with help of LIME-coverage thresholds and validation confidence. These results imply that our model can generalize and predict stably within CDD and be calibrated and have its decisions supported by evidence-based routing, yet external dataset robustness needs to be verified.

Hali tarjima qilinmagan