The Tipping Point: F-Score as a Function of the Number of Retrieved Items

Raf Guns; Christina Lioma; Birger Larsen

doi:10.1016/j.ipm.2012.02.009

The Tipping Point: F-Score as a Function of the Number of Retrieved Items

Raf Guns, Christina Lioma, Birger Larsen

Department of Communication

15 Citations (Scopus)

Abstract

One of the best known measures of information retrieval (IR) performance is the F-score, the harmonic mean of precision and recall. In this article we show that the curve of the F-score as a function of the number of retrieved items is always of the same shape: a fast concave increase to a maximum, followed by a slow decrease. In other words, there exists a single maximum, referred to as the tipping point, where the retrieval situation is 'ideal' in terms of the F-score. The tipping point thus indicates the optimal number of items to be retrieved, with more or less items resulting in a lower F-score. This empirical result is found in IR and link prediction experiments and can be partially explained theoretically, expanding on earlier results by Egghe. We discuss the implications and argue that, when comparing F-scores, one should compare the F-score curves' tipping points.

Original language	English
Journal	Information Processing & Management
Volume	48
Issue number	6
Pages (from-to)	1171
Number of pages	1,180
ISSN	0306-4573
DOIs	https://doi.org/10.1016/j.ipm.2012.02.009
Publication status	Published - Nov 2012

Keywords

Information Retrieval
Evaluation

Access to Document

10.1016/j.ipm.2012.02.009

http://www.sciencedirect.com/science/article/pii/S0306457312000295

Cite this

@article{cc61c82de2a34c46b2011549e3cedb36,

title = "The Tipping Point: F-Score as a Function of the Number of Retrieved Items",

abstract = "One of the best known measures of information retrieval (IR) performance is the F-score, the harmonic mean of precision and recall. In this article we show that the curve of the F-score as a function of the number of retrieved items is always of the same shape: a fast concave increase to a maximum, followed by a slow decrease. In other words, there exists a single maximum, referred to as the tipping point, where the retrieval situation is 'ideal' in terms of the F-score. The tipping point thus indicates the optimal number of items to be retrieved, with more or less items resulting in a lower F-score. This empirical result is found in IR and link prediction experiments and can be partially explained theoretically, expanding on earlier results by Egghe. We discuss the implications and argue that, when comparing F-scores, one should compare the F-score curves' tipping points.",

keywords = "Information Retrieval, Evaluation",

author = "Raf Guns and Christina Lioma and Birger Larsen",

year = "2012",

month = nov,

doi = "10.1016/j.ipm.2012.02.009",

language = "English",

volume = "48",

pages = "1171",

journal = "Information Processing and Management",

issn = "0306-4573",

publisher = "Elsevier",

number = "6",

}

TY - JOUR

T1 - The Tipping Point: F-Score as a Function of the Number of Retrieved Items

AU - Guns, Raf

AU - Lioma, Christina

AU - Larsen, Birger

PY - 2012/11

Y1 - 2012/11

N2 - One of the best known measures of information retrieval (IR) performance is the F-score, the harmonic mean of precision and recall. In this article we show that the curve of the F-score as a function of the number of retrieved items is always of the same shape: a fast concave increase to a maximum, followed by a slow decrease. In other words, there exists a single maximum, referred to as the tipping point, where the retrieval situation is 'ideal' in terms of the F-score. The tipping point thus indicates the optimal number of items to be retrieved, with more or less items resulting in a lower F-score. This empirical result is found in IR and link prediction experiments and can be partially explained theoretically, expanding on earlier results by Egghe. We discuss the implications and argue that, when comparing F-scores, one should compare the F-score curves' tipping points.

AB - One of the best known measures of information retrieval (IR) performance is the F-score, the harmonic mean of precision and recall. In this article we show that the curve of the F-score as a function of the number of retrieved items is always of the same shape: a fast concave increase to a maximum, followed by a slow decrease. In other words, there exists a single maximum, referred to as the tipping point, where the retrieval situation is 'ideal' in terms of the F-score. The tipping point thus indicates the optimal number of items to be retrieved, with more or less items resulting in a lower F-score. This empirical result is found in IR and link prediction experiments and can be partially explained theoretically, expanding on earlier results by Egghe. We discuss the implications and argue that, when comparing F-scores, one should compare the F-score curves' tipping points.

KW - Information Retrieval

KW - Evaluation

U2 - 10.1016/j.ipm.2012.02.009

DO - 10.1016/j.ipm.2012.02.009

M3 - Journal article

SN - 0306-4573

VL - 48

SP - 1171

JO - Information Processing and Management

JF - Information Processing and Management

IS - 6

ER -

The Tipping Point: F-Score as a Function of the Number of Retrieved Items

Abstract

Keywords

Access to Document

Fingerprint

Cite this