Developing a Test Collection for the Evaluation of Integrated Search

Marianne Lykke; Birger Larsen; Haakon Lund; Peter Ingwersen

doi:10.1007/978-3-642-12275-0_63

Developing a Test Collection for the Evaluation of Integrated Search

Marianne Lykke, Birger Larsen, Haakon Lund, Peter Ingwersen

Institut for Kommunikation

39 Citationer (Scopus)

2024 Downloads (Pure)

Abstract

The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.

Originalsprog	Engelsk
Titel	Advances in Information Retrieval : 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings
Redaktører	Cathal Gurrin, Yulan He, Gabriella Kazai, Udo Kruschwitz, Suzanne Little, Thomas Roelleke, Stefan Rüger, Keith van Rijsbergen
Vol/bind	5993
Forlag	Springer
Publikationsdato	2010
Sider	627-630
DOI	https://doi.org/10.1007/978-3-642-12275-0_63
Status	Udgivet - 2010
Begivenhed	European Conference on IR Research - Milton Keynes, Storbritannien Varighed: 28 mar. 2010 → 31 maj 2010 Konferencens nummer: 32

Konference

Konference	European Conference on IR Research
Nummer	32
Land/Område	Storbritannien
By	Milton Keynes
Periode	28/03/2010 → 31/05/2010

Navn	Lecture notes in computer science
Nummer	5993
ISSN	0302-9743

Emneord

Design af testsamlinger
Integreret søgning

Adgang til dokumentet

10.1007/978-3-642-12275-0_63

ECIR poster2010 finalIndsendt manuskript, 3,28 MB

Citationsformater

Lykke, M., Larsen, B., Lund, H., & Ingwersen, P. (2010). Developing a Test Collection for the Evaluation of Integrated Search. I C. Gurrin, Y. He, G. Kazai, U. Kruschwitz, S. Little, T. Roelleke, S. Rüger, & K. van Rijsbergen (red.), Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings (Bind 5993, s. 627-630). Springer. Lecture notes in computer science Nr. 5993 https://doi.org/10.1007/978-3-642-12275-0_63

Developing a Test Collection for the Evaluation of Integrated Search. / Lykke, Marianne; Larsen, Birger; Lund, Haakon et al.

Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings. red. / Cathal Gurrin; Yulan He; Gabriella Kazai; Udo Kruschwitz; Suzanne Little; Thomas Roelleke; Stefan Rüger; Keith van Rijsbergen. Bind 5993 Springer, 2010. s. 627-630 (Lecture notes in computer science; Nr. 5993).

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

Lykke, M, Larsen, B, Lund, H & Ingwersen, P 2010, Developing a Test Collection for the Evaluation of Integrated Search. i C Gurrin, Y He, G Kazai, U Kruschwitz, S Little, T Roelleke, S Rüger & K van Rijsbergen (red), Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings. bind 5993, Springer, Lecture notes in computer science, nr. 5993, s. 627-630, European Conference on IR Research, Milton Keynes, Storbritannien, 28/03/2010. https://doi.org/10.1007/978-3-642-12275-0_63

Lykke M, Larsen B, Lund H, Ingwersen P. Developing a Test Collection for the Evaluation of Integrated Search. I Gurrin C, He Y, Kazai G, Kruschwitz U, Little S, Roelleke T, Rüger S, van Rijsbergen K, red., Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings. Bind 5993. Springer. 2010. s. 627-630. (Lecture notes in computer science; Nr. 5993). doi: 10.1007/978-3-642-12275-0_63

Lykke, Marianne ; Larsen, Birger ; Lund, Haakon et al. / Developing a Test Collection for the Evaluation of Integrated Search. Advances in Information Retrieval: 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings. red. / Cathal Gurrin ; Yulan He ; Gabriella Kazai ; Udo Kruschwitz ; Suzanne Little ; Thomas Roelleke ; Stefan Rüger ; Keith van Rijsbergen. Bind 5993 Springer, 2010. s. 627-630 (Lecture notes in computer science; Nr. 5993).

@inproceedings{3ac135a4da2e46ff8444e6d295a6252a,

title = "Developing a Test Collection for the Evaluation of Integrated Search",

abstract = "The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.",

keywords = "Design af testsamlinger, Integreret s{\o}gning, Test collection design, Task-based IR, Integrated search",

author = "Marianne Lykke and Birger Larsen and Haakon Lund and Peter Ingwersen",

year = "2010",

doi = "10.1007/978-3-642-12275-0_63",

language = "English",

volume = "5993",

series = "Lecture notes in computer science",

publisher = "Springer",

number = "5993",

pages = "627--630",

editor = "Cathal Gurrin and Yulan He and Gabriella Kazai and Udo Kruschwitz and Suzanne Little and Thomas Roelleke and Stefan R{\"u}ger and {van Rijsbergen}, Keith",

booktitle = "Advances in Information Retrieval",

note = "European Conference on IR Research ; Conference date: 28-03-2010 Through 31-05-2010",

}

TY - GEN

T1 - Developing a Test Collection for the Evaluation of Integrated Search

AU - Lykke, Marianne

AU - Larsen, Birger

AU - Lund, Haakon

AU - Ingwersen, Peter

N1 - Conference code: 32

PY - 2010

Y1 - 2010

N2 - The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.

AB - The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.

KW - Design af testsamlinger

KW - Integreret søgning

KW - Test collection design

KW - Task-based IR

KW - Integrated search

U2 - 10.1007/978-3-642-12275-0_63

DO - 10.1007/978-3-642-12275-0_63

M3 - Article in proceedings

VL - 5993

T3 - Lecture notes in computer science

SP - 627

EP - 630

BT - Advances in Information Retrieval

A2 - Gurrin, Cathal

A2 - He, Yulan

A2 - Kazai, Gabriella

A2 - Kruschwitz, Udo

A2 - Little, Suzanne

A2 - Roelleke, Thomas

A2 - Rüger, Stefan

A2 - van Rijsbergen, Keith

PB - Springer

T2 - European Conference on IR Research

Y2 - 28 March 2010 through 31 May 2010

ER -

Developing a Test Collection for the Evaluation of Integrated Search

Abstract

Konference

Emneord

Adgang til dokumentet

Fingeraftryk

Citationsformater