Developing a Test Collection for the Evaluation of Integrated Search

Marianne Lykke; Birger Larsen; Haakon Lund; Peter Ingwersen

doi:10.1007/978-3-642-12275-0_63

Developing a Test Collection for the Evaluation of Integrated Search

Marianne Lykke, Birger Larsen, Haakon Lund, Peter Ingwersen

Department of Communication

39 Citations (Scopus)

2024 Downloads (Pure)

Abstract

The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.

Original language	English
Title of host publication	Advances in Information Retrieval : 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings
Editors	Cathal Gurrin, Yulan He, Gabriella Kazai, Udo Kruschwitz, Suzanne Little, Thomas Roelleke, Stefan Rüger, Keith van Rijsbergen
Volume	5993
Publisher	Springer
Publication date	2010
Pages	627-630
DOIs	https://doi.org/10.1007/978-3-642-12275-0_63
Publication status	Published - 2010
Event	European Conference on IR Research - Milton Keynes, United Kingdom Duration: 28 Mar 2010 → 31 May 2010 Conference number: 32

Conference

Conference	European Conference on IR Research
Number	32
Country/Territory	United Kingdom
City	Milton Keynes
Period	28/03/2010 → 31/05/2010

Series	Lecture notes in computer science
Number	5993
ISSN	0302-9743

Keywords

Test collection design
Task-based IR
Integrated search

Access to Document

10.1007/978-3-642-12275-0_63

ECIR poster2010 finalSubmitted manuscript, 3.28 MB

Cite this

Lykke, M., Larsen, B., Lund, H., & Ingwersen, P. (2010). Developing a Test Collection for the Evaluation of Integrated Search. In C. Gurrin, Y. He, G. Kazai, U. Kruschwitz, S. Little, T. Roelleke, S. Rüger, & K. van Rijsbergen (Eds.), Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings (Vol. 5993, pp. 627-630). Springer. Lecture notes in computer science No. 5993 https://doi.org/10.1007/978-3-642-12275-0_63

Developing a Test Collection for the Evaluation of Integrated Search. / Lykke, Marianne; Larsen, Birger; Lund, Haakon et al.

Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings. ed. / Cathal Gurrin; Yulan He; Gabriella Kazai; Udo Kruschwitz; Suzanne Little; Thomas Roelleke; Stefan Rüger; Keith van Rijsbergen. Vol. 5993 Springer, 2010. p. 627-630 (Lecture notes in computer science; No. 5993).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Lykke, M, Larsen, B, Lund, H & Ingwersen, P 2010, Developing a Test Collection for the Evaluation of Integrated Search. in C Gurrin, Y He, G Kazai, U Kruschwitz, S Little, T Roelleke, S Rüger & K van Rijsbergen (eds), Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings. vol. 5993, Springer, Lecture notes in computer science, no. 5993, pp. 627-630, European Conference on IR Research, Milton Keynes, United Kingdom, 28/03/2010. https://doi.org/10.1007/978-3-642-12275-0_63

Lykke M, Larsen B, Lund H, Ingwersen P. Developing a Test Collection for the Evaluation of Integrated Search. In Gurrin C, He Y, Kazai G, Kruschwitz U, Little S, Roelleke T, Rüger S, van Rijsbergen K, editors, Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings. Vol. 5993. Springer. 2010. p. 627-630. (Lecture notes in computer science; No. 5993). doi: 10.1007/978-3-642-12275-0_63

Lykke, Marianne ; Larsen, Birger ; Lund, Haakon et al. / Developing a Test Collection for the Evaluation of Integrated Search. Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings. editor / Cathal Gurrin ; Yulan He ; Gabriella Kazai ; Udo Kruschwitz ; Suzanne Little ; Thomas Roelleke ; Stefan Rüger ; Keith van Rijsbergen. Vol. 5993 Springer, 2010. pp. 627-630 (Lecture notes in computer science; No. 5993).

@inproceedings{3ac135a4da2e46ff8444e6d295a6252a,

title = "Developing a Test Collection for the Evaluation of Integrated Search",

abstract = "The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.",

keywords = "Design af testsamlinger, Integreret s{\o}gning, Test collection design, Task-based IR, Integrated search",

author = "Marianne Lykke and Birger Larsen and Haakon Lund and Peter Ingwersen",

year = "2010",

doi = "10.1007/978-3-642-12275-0_63",

language = "English",

volume = "5993",

series = "Lecture notes in computer science",

publisher = "Springer",

number = "5993",

pages = "627--630",

editor = "Cathal Gurrin and Yulan He and Gabriella Kazai and Udo Kruschwitz and Suzanne Little and Thomas Roelleke and Stefan R{\"u}ger and {van Rijsbergen}, Keith",

booktitle = "Advances in Information Retrieval",

note = "European Conference on IR Research ; Conference date: 28-03-2010 Through 31-05-2010",

}

TY - GEN

T1 - Developing a Test Collection for the Evaluation of Integrated Search

AU - Lykke, Marianne

AU - Larsen, Birger

AU - Lund, Haakon

AU - Ingwersen, Peter

N1 - Conference code: 32

PY - 2010

Y1 - 2010

N2 - The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.

AB - The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.

KW - Design af testsamlinger

KW - Integreret søgning

KW - Test collection design

KW - Task-based IR

KW - Integrated search

U2 - 10.1007/978-3-642-12275-0_63

DO - 10.1007/978-3-642-12275-0_63

M3 - Article in proceedings

VL - 5993

T3 - Lecture notes in computer science

SP - 627

EP - 630

BT - Advances in Information Retrieval

A2 - Gurrin, Cathal

A2 - He, Yulan

A2 - Kazai, Gabriella

A2 - Kruschwitz, Udo

A2 - Little, Suzanne

A2 - Roelleke, Thomas

A2 - Rüger, Stefan

A2 - van Rijsbergen, Keith

PB - Springer

T2 - European Conference on IR Research

Y2 - 28 March 2010 through 31 May 2010

ER -

Developing a Test Collection for the Evaluation of Integrated Search

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this