RankEval: An Evaluation and Analysis Framework for Learning-to-Rank Solutions
Accepted at SIGIR ’17: ACM Conference on Research and Development in Information Retrieval .
Abstract. In this demo paper we propose RankEval, an open-source tool for the analysis and evaluation of Learning-to-Rank (LtR) models based on ensembles of regression trees. Gradient Boosted Regression Trees (GBRT) is a ﬂexible statistical learning technique for classiﬁcation and regression at the state of the art for training eﬀective LtR solutions. Indeed, the success of GBRT fostered the development of several open-source LtR libraries targeting eﬃciency of the learning phase and eﬀectiveness of the resulting models. However, these libraries oﬀer only very limited help for the tuning and evaluation of the trained models. In addition, the implementations provided for even the most traditional IR evaluation metrics diﬀer from library to library, thus making the objective evaluation and comparison between trained models a diﬃcult task. RankEval addresses these issues by providing a common ground for LtR libraries that oﬀers useful and interoperable tools for a comprehensive comparison and in-depth analysis of ranking models.
 Claudio Lucchese, Cristina Ioana Muntean, Franco Maria Nardini, Raﬀaele Perego, and Salvatore Trani. Rankeval: An evaluation and analysis framework for learning-to-rank solutions. In SIGIR ’17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017.