On the Evaluation of Entity Profiles

Maarten de Rijke; Krisztian Balog; Toine Bogers; Antal van den Bosch

On the Evaluation of Entity Profiles

Maarten de Rijke, Krisztian Balog, Toine Bogers, Antal van den Bosch

Institut for Kommunikation

4 Citationer (Scopus)

238 Downloads (Pure)

Abstract

Entity profiling is the task of identifying and ranking descriptions of a given entity. The task may be viewed as one where the descriptions being sought are terms that need to be selected from a knowledge source (such as an ontology or thesaurus). In this case, entity profiling systems can be assessed by means of precision and recall values of the descriptive terms produced. However, recent evidence suggests that more sophisticated metrics are needed that go beyond mere lexical matching of system-produced descriptors against a ground truth, allowing for graded relevance and rewarding diversity in the list of descriptors returned. In this note, we motivate and propose such a metric.

Originalsprog	Engelsk
Publikationsdato	2010
Antal sider	6
Status	Udgivet - 2010
Begivenhed	Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2010) - Padua, Italien Varighed: 20 sep. 2010 → 23 sep. 2010

Konference

Konference	Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2010)
Land/Område	Italien
By	Padua
Periode	20/09/2010 → 23/09/2010

Adgang til dokumentet

On the Evaluating of Entity Profiles.pdfAccepteret manuskript, 118 KB

Citationsformater

@conference{cd9f71f916a8435991023808fbad2744,

title = "On the Evaluation of Entity Profiles",

abstract = "Entity profiling is the task of identifying and ranking descriptions of a given entity. The task may be viewed as one where the descriptions being sought are terms that need to be selected from a knowledge source (such as an ontology or thesaurus). In this case, entity profiling systems can be assessed by means of precision and recall values of the descriptive terms produced. However, recent evidence suggests that more sophisticated metrics are needed that go beyond mere lexical matching of system-produced descriptors against a ground truth, allowing for graded relevance and rewarding diversity in the list of descriptors returned. In this note, we motivate and propose such a metric.",

keywords = "entity profiling, evaluation, entity retrieval, entity search",

author = "{de Rijke}, Maarten and Krisztian Balog and Toine Bogers and {van den Bosch}, Antal",

year = "2010",

language = "English",

note = "Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2010) ; Conference date: 20-09-2010 Through 23-09-2010",

}

TY - CONF

T1 - On the Evaluation of Entity Profiles

AU - de Rijke, Maarten

AU - Balog, Krisztian

AU - Bogers, Toine

AU - van den Bosch, Antal

PY - 2010

Y1 - 2010

N2 - Entity profiling is the task of identifying and ranking descriptions of a given entity. The task may be viewed as one where the descriptions being sought are terms that need to be selected from a knowledge source (such as an ontology or thesaurus). In this case, entity profiling systems can be assessed by means of precision and recall values of the descriptive terms produced. However, recent evidence suggests that more sophisticated metrics are needed that go beyond mere lexical matching of system-produced descriptors against a ground truth, allowing for graded relevance and rewarding diversity in the list of descriptors returned. In this note, we motivate and propose such a metric.

AB - Entity profiling is the task of identifying and ranking descriptions of a given entity. The task may be viewed as one where the descriptions being sought are terms that need to be selected from a knowledge source (such as an ontology or thesaurus). In this case, entity profiling systems can be assessed by means of precision and recall values of the descriptive terms produced. However, recent evidence suggests that more sophisticated metrics are needed that go beyond mere lexical matching of system-produced descriptors against a ground truth, allowing for graded relevance and rewarding diversity in the list of descriptors returned. In this note, we motivate and propose such a metric.

KW - entity profiling

KW - evaluation

KW - entity retrieval

KW - entity search

M3 - Paper

T2 - Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2010)

Y2 - 20 September 2010 through 23 September 2010

ER -

On the Evaluation of Entity Profiles

Abstract

Konference

Adgang til dokumentet

Fingeraftryk

Citationsformater