Assessors' Search Result Satisfaction Associated with Relevance in a Scientific Domain

Peter Ingwersen; Marianne Lykke; Toine Bogers; Birger Larsen; Haakon Lund

doi:10.1145/1840784.1840826

Assessors' Search Result Satisfaction Associated with Relevance in a Scientific Domain

Peter Ingwersen, Marianne Lykke, Toine Bogers, Birger Larsen, Haakon Lund

Department of Communication

1 Citation (Scopus)

607 Downloads (Pure)

Abstract

In this poster we investigate the associations between perceived ease of assessment of situational relevance made by a four-point scale, perceived satisfaction with retrieval results and the actual relevance assessments and retrieval performance made by test collection assessors based on their own genuine information tasks. Ease of assessment and search satisfaction are cross tabulated with retrieval performance measured by Normalized Discounted Cumulated Gain. Results show that when assessors find small numbers of relevant documents they tend to regard the search results with dissatisfaction and, in addition, they obtain lower performance for all document types involved, except for monographic records.

Original language	English
Publication date	2010
Number of pages	5
DOIs	https://doi.org/10.1145/1840784.1840826
Publication status	Published - 2010

Access to Document

10.1145/1840784.1840826

PreprintSubmitted manuscript, 960 KB

http://portal.acm.org/citation.cfm?id=1840826&CFID=109504018&CFTOKEN=77447240

Cite this

@conference{9ce4de35aa3b44708903e01329b279f9,

title = "Assessors' Search Result Satisfaction Associated with Relevance in a Scientific Domain",

abstract = "In this poster we investigate the associations between perceived ease of assessment of situational relevance made by a four-point scale, perceived satisfaction with retrieval results and the actual relevance assessments and retrieval performance made by test collection assessors based on their own genuine information tasks. Ease of assessment and search satisfaction are cross tabulated with retrieval performance measured by Normalized Discounted Cumulated Gain. Results show that when assessors find small numbers of relevant documents they tend to regard the search results with dissatisfaction and, in addition, they obtain lower performance for all document types involved, except for monographic records.",

author = "Peter Ingwersen and Marianne Lykke and Toine Bogers and Birger Larsen and Haakon Lund",

note = "In: Belkin, N. J.Kelly, D.(reds). IIiX'10 Proceeding of the Third Symposium on Information Interaction in Context, New Brunswick, NJ, USA, August 18-21, 2010, p. 283-287. (ISBN: 978-1-4503-0247-0)",

year = "2010",

doi = "10.1145/1840784.1840826",

language = "English",

}

TY - CONF

T1 - Assessors' Search Result Satisfaction Associated with Relevance in a Scientific Domain

AU - Ingwersen, Peter

AU - Lykke, Marianne

AU - Bogers, Toine

AU - Larsen, Birger

AU - Lund, Haakon

N1 - In: Belkin, N. J.Kelly, D.(reds). IIiX'10 Proceeding of the Third Symposium on Information Interaction in Context, New Brunswick, NJ, USA, August 18-21, 2010, p. 283-287. (ISBN: 978-1-4503-0247-0)

PY - 2010

Y1 - 2010

N2 - In this poster we investigate the associations between perceived ease of assessment of situational relevance made by a four-point scale, perceived satisfaction with retrieval results and the actual relevance assessments and retrieval performance made by test collection assessors based on their own genuine information tasks. Ease of assessment and search satisfaction are cross tabulated with retrieval performance measured by Normalized Discounted Cumulated Gain. Results show that when assessors find small numbers of relevant documents they tend to regard the search results with dissatisfaction and, in addition, they obtain lower performance for all document types involved, except for monographic records.

AB - In this poster we investigate the associations between perceived ease of assessment of situational relevance made by a four-point scale, perceived satisfaction with retrieval results and the actual relevance assessments and retrieval performance made by test collection assessors based on their own genuine information tasks. Ease of assessment and search satisfaction are cross tabulated with retrieval performance measured by Normalized Discounted Cumulated Gain. Results show that when assessors find small numbers of relevant documents they tend to regard the search results with dissatisfaction and, in addition, they obtain lower performance for all document types involved, except for monographic records.

U2 - 10.1145/1840784.1840826

DO - 10.1145/1840784.1840826

M3 - Poster

ER -