An investigation into the use of document scores for optimisation over rank-biased precision

Research output: Chapter in Book / Conference PaperConference Paperpeer-review

Abstract

![CDATA[When a Document Retrieval system receives a query, a Relevance model is used to provide a score to each document based on its relevance to the query. Relevance models have parameters that should be tuned to optimise the accuracy of the relevance model for the document set and expected queries, where the accuracy is computed using an Information Retrieval evaluation function. Unfortunately, evaluation functions contain a discontinuous mapping from the document scores to document ranks, making optimisation of relevance models difficult using gradient based optimisation methods. In this article, we identify that the evaluation function Rank-biased Precision (RBP) performs a conversion from document scores, to ranks, then to weights. Therefore, we investigate the utility of bypassing the conversion to ranks (converting document score directly to RBP weights) for Relevance model tuning purposes. We find that using transformed BM25 document scores in the place of the RBP weights provides an equivalent optimisation function for mean and median RBP. Therefore, we can use this document score based RBP as a surrogate for tuning relevance models.]]
Original languageEnglish
Title of host publicationInformation Retrieval Technology: Proceedings of the 13th Asia Information Retrieval Societies Conference, AIRS 2017, Jeju Island, South Korea, 22-24 November 2017
PublisherSpringer
Pages197-209
Number of pages13
ISBN (Print)9783319701448
DOIs
Publication statusPublished - 2017
EventAsia Information Retrieval Societies Conference -
Duration: 22 Nov 2017 → …

Publication series

Name
ISSN (Print)0302-9743

Conference

ConferenceAsia Information Retrieval Societies Conference
Period22/11/17 → …

Keywords

  • information retrieval
  • relevance

Fingerprint

Dive into the research topics of 'An investigation into the use of document scores for optimisation over rank-biased precision'. Together they form a unique fingerprint.

Cite this