Asosiy kontentga oʻtish
Boshqa

Code and results for: Predicting Change Is Not Predicting Improvement — A Leakage-Free Study of Confidence-Calibrated Selective Reranking in Hybrid Retrieval

Rustamjon NasridinovTashkent University of Information Technologies named after Muhammad al-Khwarizmi
2026en
Ochiq kirishYashil

Litsenziya: cc-by

ABI

Annotatsiya

Code and per-query results accompanying the paper "Predicting Change Is Not Predicting Improvement: A Leakage-Free Study of Confidence-Calibrated Selective Reranking in Hybrid Retrieval" (submitted to the Journal of Intelligent Information Systems). The package contains six Google Colab notebooks (and equivalent single-cell scripts) that implement the full pipeline: BM25 and multilingual-E5 hybrid retrieval with static fusion, query-aware LambdaMART, cross-encoder reranking (bge-reranker-v2-m3), confidence calibration, development- and train-fitted routing gates with query performance predictors and a reranker probe, zero-shot evaluation on four BEIR collections, judgment-sparsity analysis (judged@10, condensed nDCG@10) and the change-versus-direction decomposition. Experiments follow a leakage-free protocol: training queries fit models, development queries select hyperparameters and budgets, and test queries are used once. The pre-registered success criterion for per-query routing is reported as not met. Collections: SciFact, NFCorpus, FiQA-2018 (in-domain); SciDocs, ArguAna, TREC-COVID, Touché-2020 (zero-shot). BEIR data and model weights are downloaded from their original sources and are not redistributed.

Hali tarjima qilinmagan

Identifikatorlar

Iqtiboslar va manbalar

0 ta iqtibos0 ta foydalanilgan manba