Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective

Chariste Papaeconomou; Annemarie F. Zijlema; Peter Ingwersen

Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective

Chariste Papaeconomou, Annemarie F. Zijlema, Peter Ingwersen

Department of Communication

13 Citations (Scopus)

Abstract

The paper presents the results of a case study of searcher's relevance criteria used for assessments of Web pages in a perspective of learning style. 15 test persons participated in the experiments based on two simulated work tasks that provided cover stories to trigger their information needs. Two learning styles were examined: Global and Sequential learners. The study applied eye-tracking for the observation of relevance hot spots on Web pages, learning style index analysis and post-search interviews to gain more in-depth information on relevance behavior. Findings reveal that with respect to use of graded relevance scores and number of relevance criteria applied per task and test person there are no significant difference between the different styles. Although there differences are detected in the use of relevance criteria between Global and Sequential learners during assessments, they are statistically insignificant. When interviewed in retrospective the resulting profiles tend to become even similar across learning styles but a shift occurs from instant assessments with content features of web pages replacing topicality judgments as predominant relevance criteria.

Original language	English
Title of host publication	Information Interaction in Context : Proceedings of the second IIiX Symposium on Information Interaction in Context
Editors	Pia Borlund, Jesper Schneider, Mounia Lalmas
Number of pages	8
Publisher	ACM Press
Publication date	2008
Pages	123-130
ISBN (Print)	978-1-60558-310-5
Publication status	Published - 2008
Event	Second IIiX Symposium on Information Interaction in Context - London, United Kingdom Duration: 14 Oct 2008 → 17 Oct 2008 Conference number: 2

Conference

Conference	Second IIiX Symposium on Information Interaction in Context
Number	2
Country/Territory	United Kingdom
City	London
Period	14/10/2008 → 17/10/2008

Series	ACM International Conference Proceedings Series

Keywords

Information retrieval
Seeking behaviou
Web collection
Relevance assessments

Access to Document

http://portal.acm.org/citation.cfm?id=1414694.1414722&coll=ACM&dl=ACM&CFID=33837190&CFTOKEN=43473685

Cite this

Papaeconomou, C., Zijlema, A. F., & Ingwersen, P. (2008). Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective. In P. Borlund, J. Schneider, & M. Lalmas (Eds.), Information Interaction in Context: Proceedings of the second IIiX Symposium on Information Interaction in Context (pp. 123-130). ACM Press. ACM International Conference Proceedings Series http://portal.acm.org/citation.cfm?id=1414694.1414722&coll=ACM&dl=ACM&CFID=33837190&CFTOKEN=43473685

Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective. / Papaeconomou, Chariste; Zijlema, Annemarie F.; Ingwersen, Peter.

Information Interaction in Context: Proceedings of the second IIiX Symposium on Information Interaction in Context. ed. / Pia Borlund; Jesper Schneider; Mounia Lalmas. ACM Press, 2008. p. 123-130 (ACM International Conference Proceedings Series).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Papaeconomou, C, Zijlema, AF & Ingwersen, P 2008, Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective. in P Borlund, J Schneider & M Lalmas (eds), Information Interaction in Context: Proceedings of the second IIiX Symposium on Information Interaction in Context. ACM Press, ACM International Conference Proceedings Series, pp. 123-130, Second IIiX Symposium on Information Interaction in Context, London, United Kingdom, 14/10/2008. <http://portal.acm.org/citation.cfm?id=1414694.1414722&coll=ACM&dl=ACM&CFID=33837190&CFTOKEN=43473685>

Papaeconomou C, Zijlema AF, Ingwersen P. Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective. In Borlund P, Schneider J, Lalmas M, editors, Information Interaction in Context: Proceedings of the second IIiX Symposium on Information Interaction in Context. ACM Press. 2008. p. 123-130. (ACM International Conference Proceedings Series).

Papaeconomou, Chariste ; Zijlema, Annemarie F. ; Ingwersen, Peter. / Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective. Information Interaction in Context: Proceedings of the second IIiX Symposium on Information Interaction in Context. editor / Pia Borlund ; Jesper Schneider ; Mounia Lalmas. ACM Press, 2008. pp. 123-130 (ACM International Conference Proceedings Series).

@inproceedings{1f700cba9e0849dbb074f4a7ff69ef34,

title = "Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective",

abstract = "The paper presents the results of a case study of searcher's relevance criteria used for assessments of Web pages in a perspective of learning style. 15 test persons participated in the experiments based on two simulated work tasks that provided cover stories to trigger their information needs. Two learning styles were examined: Global and Sequential learners. The study applied eye-tracking for the observation of relevance hot spots on Web pages, learning style index analysis and post-search interviews to gain more in-depth information on relevance behavior. Findings reveal that with respect to use of graded relevance scores and number of relevance criteria applied per task and test person there are no significant difference between the different styles. Although there differences are detected in the use of relevance criteria between Global and Sequential learners during assessments, they are statistically insignificant. When interviewed in retrospective the resulting profiles tend to become even similar across learning styles but a shift occurs from instant assessments with content features of web pages replacing topicality judgments as predominant relevance criteria. ",

keywords = "Informationss{\o}gning, S{\o}geadf{\ae}rd, Websider, Relevanskriterier, Information retrieval, Seeking behaviou, Web collection, Relevance assessments",

author = "Chariste Papaeconomou and Zijlema, {Annemarie F.} and Peter Ingwersen",

year = "2008",

language = "English",

isbn = "978-1-60558-310-5",

series = "ACM International Conference Proceedings Series",

publisher = "ACM Press",

pages = "123--130",

editor = "Pia Borlund and Jesper Schneider and Mounia Lalmas",

booktitle = "Information Interaction in Context",

note = "Second IIiX Symposium on Information Interaction in Context ; Conference date: 14-10-2008 Through 17-10-2008",

}

TY - GEN

T1 - Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective

AU - Papaeconomou, Chariste

AU - Zijlema, Annemarie F.

AU - Ingwersen, Peter

N1 - Conference code: 2

PY - 2008

Y1 - 2008

N2 - The paper presents the results of a case study of searcher's relevance criteria used for assessments of Web pages in a perspective of learning style. 15 test persons participated in the experiments based on two simulated work tasks that provided cover stories to trigger their information needs. Two learning styles were examined: Global and Sequential learners. The study applied eye-tracking for the observation of relevance hot spots on Web pages, learning style index analysis and post-search interviews to gain more in-depth information on relevance behavior. Findings reveal that with respect to use of graded relevance scores and number of relevance criteria applied per task and test person there are no significant difference between the different styles. Although there differences are detected in the use of relevance criteria between Global and Sequential learners during assessments, they are statistically insignificant. When interviewed in retrospective the resulting profiles tend to become even similar across learning styles but a shift occurs from instant assessments with content features of web pages replacing topicality judgments as predominant relevance criteria.

AB - The paper presents the results of a case study of searcher's relevance criteria used for assessments of Web pages in a perspective of learning style. 15 test persons participated in the experiments based on two simulated work tasks that provided cover stories to trigger their information needs. Two learning styles were examined: Global and Sequential learners. The study applied eye-tracking for the observation of relevance hot spots on Web pages, learning style index analysis and post-search interviews to gain more in-depth information on relevance behavior. Findings reveal that with respect to use of graded relevance scores and number of relevance criteria applied per task and test person there are no significant difference between the different styles. Although there differences are detected in the use of relevance criteria between Global and Sequential learners during assessments, they are statistically insignificant. When interviewed in retrospective the resulting profiles tend to become even similar across learning styles but a shift occurs from instant assessments with content features of web pages replacing topicality judgments as predominant relevance criteria.

KW - Informationssøgning

KW - Søgeadfærd

KW - Websider

KW - Relevanskriterier

KW - Information retrieval

KW - Seeking behaviou

KW - Web collection

KW - Relevance assessments

M3 - Article in proceedings

SN - 978-1-60558-310-5

T3 - ACM International Conference Proceedings Series

SP - 123

EP - 130

BT - Information Interaction in Context

A2 - Borlund, Pia

A2 - Schneider, Jesper

A2 - Lalmas, Mounia

PB - ACM Press

T2 - Second IIiX Symposium on Information Interaction in Context

Y2 - 14 October 2008 through 17 October 2008

ER -