Ingredients and meals rather than recipes: a proposal for research that does not treat usability evaluation methods as indivisible wholes

TY - JOUR

T1 - Ingredients and meals rather than recipes

T2 - a proposal for research that does not treat usability evaluation methods as indivisible wholes

AU - Woolrych, Alan

AU - Hornbæk, Kasper

AU - Frøkjær, Erik

AU - Cockton, Gilbert

PY - 2011/10

Y1 - 2011/10

N2 - To better support usability practice, most usability research focuses on evaluation methods. New ideas in usability research are mostly proposed as new evaluation methods. Many publications describe experiments that compare methods. Comparisons may indicate that some methods have important deficiencies, and thus often advise usability practitioners to prefer a specific method in a particular situation. An expectation persists in human–computer interaction (HCI) that results about evaluation methods should be the standard “unit of contribution” rather than favoring larger units (e.g., usability work as a whole) or smaller ones (e.g., the impact of specific aspects of a method). This article argues that these foci on comparisons and method innovations ignore the reality that usability evaluation methods are loose incomplete collections of resources, which successful practitioners configure, adapt, and complement to match specific project circumstances. Through a review of existing research on methods and resources, resources associated with specific evaluation methods, and ones that can complement existing methods, or be used separately, are identified. Next, a generic classification scheme for evaluation resources is developed, and the scheme is extended with project specific resources that impact the effective use of methods. With these reviews and analyses in place, implications for research, teaching, and practice are derived. Throughout, the article draws on culinary analogies. A recipe is nothing without its ingredients, and just as the quality of what is cooked reflects the quality of its ingredients, so too does the quality of usability work reflect the quality of resources as configured and combined. A method, like a recipe, is at best a guide to action for those adopting approaches to usability that are new to them. As with culinary dishes, HCI needs to focus more on what gets cooked, and how it gets cooked, and not just on how recipes suggest that it could be cooked.

AB - To better support usability practice, most usability research focuses on evaluation methods. New ideas in usability research are mostly proposed as new evaluation methods. Many publications describe experiments that compare methods. Comparisons may indicate that some methods have important deficiencies, and thus often advise usability practitioners to prefer a specific method in a particular situation. An expectation persists in human–computer interaction (HCI) that results about evaluation methods should be the standard “unit of contribution” rather than favoring larger units (e.g., usability work as a whole) or smaller ones (e.g., the impact of specific aspects of a method). This article argues that these foci on comparisons and method innovations ignore the reality that usability evaluation methods are loose incomplete collections of resources, which successful practitioners configure, adapt, and complement to match specific project circumstances. Through a review of existing research on methods and resources, resources associated with specific evaluation methods, and ones that can complement existing methods, or be used separately, are identified. Next, a generic classification scheme for evaluation resources is developed, and the scheme is extended with project specific resources that impact the effective use of methods. With these reviews and analyses in place, implications for research, teaching, and practice are derived. Throughout, the article draws on culinary analogies. A recipe is nothing without its ingredients, and just as the quality of what is cooked reflects the quality of its ingredients, so too does the quality of usability work reflect the quality of resources as configured and combined. A method, like a recipe, is at best a guide to action for those adopting approaches to usability that are new to them. As with culinary dishes, HCI needs to focus more on what gets cooked, and how it gets cooked, and not just on how recipes suggest that it could be cooked.

U2 - 10.1080/10447318.2011.555314

DO - 10.1080/10447318.2011.555314

M3 - Journal article

SN - 2180-1347

VL - 27

SP - 940

EP - 970

JO - International Journal of Human Computer Interaction

JF - International Journal of Human Computer Interaction

IS - 10

ER -