Uncertainty in Rank-Biased Precision

Research output: Chapter in Book / Conference PaperConference Paperpeer-review

2 Citations (Scopus)

Abstract

![CDATA[Information retrieval metrics that provide uncertainty intervals when faced with unjudged documents, such as Rank-Biased Precision (RBP), provide us with an indication of the upper and lower bound of the system score. Unfortunately, the uncertainty is disregarded when examining the mean over a set of queries. In this article, we examine the distribution of the uncertainty per query and averaged over all queries, under the assumption that each unjudged document has the same probability of being relevant. We also derive equations for the mean, variance, and distribution of Mean RBP uncertainty. Finally, the impact of our assumption is assessed using simulation. We find that by removing the assumption of equal probability of relevance, we obtain a scaled form of the previously defined mean and standard deviation for the distribution of Mean RBP uncertainty.]]
Original languageEnglish
Title of host publicationADCS 2016: Proceedings of the 21st Australasian Document Computing Symposium, Caulfield, Victoria, Australia, December 6-7, 2016
PublisherACM
Pages73-76
Number of pages4
ISBN (Print)9781450348652
DOIs
Publication statusPublished - 2016
EventAustralasian Document Computing Symposium -
Duration: 6 Dec 2016 → …

Conference

ConferenceAustralasian Document Computing Symposium
Period6/12/16 → …

Keywords

  • algorithms
  • information retrieval
  • search engines
  • uncertainty

Fingerprint

Dive into the research topics of 'Uncertainty in Rank-Biased Precision'. Together they form a unique fingerprint.

Cite this