TY - JOUR
T1 - Evaluating a scientific collaboratory
T2 - Results of a controlled experiment
AU - Sonnenwald, Diane H.
AU - Whitton, Mary C.
AU - Maglaughlin, Kelly L.
PY - 2003/6/1
Y1 - 2003/6/1
N2 - The evaluation of scientific collaboratories has lagged behind their development. Do the capabilities afforded by collaboratories outweigh their disadvantages? To evaluate a scientific collaboratory system, we conducted a repeated-measures controlled experiment that compared the outcomes and process of scientific work completed by 20 pairs of participants (upper level undergraduate science students) working face-to-face and remotely. We collected scientific outcomes (graded lab reports) to investigate the quality of scientific work, post-questionnaire data to measure the adoptability of the system, and post-interviews to understand the participants' views of doing science under both conditions. We hypothesized that study participants would be less effective, report more difficulty, and be less favorably inclined to adopt the system when collaborating remotely. Contrary to expectations, the quantitative data showed no statistically significant differences with respect to effectiveness and adoption.The qualitative data helped explain this null result: participants reported advantages and disadvantages working under both conditions and developed work-arounds to cope with the perceived disadvantages of collaborating remotely. While the data analysis produced null results, considered as a whole, the analysis leads us to conclude there is positive potential for the development and adoption of scientific collaboratory systems.
AB - The evaluation of scientific collaboratories has lagged behind their development. Do the capabilities afforded by collaboratories outweigh their disadvantages? To evaluate a scientific collaboratory system, we conducted a repeated-measures controlled experiment that compared the outcomes and process of scientific work completed by 20 pairs of participants (upper level undergraduate science students) working face-to-face and remotely. We collected scientific outcomes (graded lab reports) to investigate the quality of scientific work, post-questionnaire data to measure the adoptability of the system, and post-interviews to understand the participants' views of doing science under both conditions. We hypothesized that study participants would be less effective, report more difficulty, and be less favorably inclined to adopt the system when collaborating remotely. Contrary to expectations, the quantitative data showed no statistically significant differences with respect to effectiveness and adoption.The qualitative data helped explain this null result: participants reported advantages and disadvantages working under both conditions and developed work-arounds to cope with the perceived disadvantages of collaborating remotely. While the data analysis produced null results, considered as a whole, the analysis leads us to conclude there is positive potential for the development and adoption of scientific collaboratory systems.
KW - Collaboration
KW - Controlled experiment
KW - Geographically-distributed work
KW - Nanoscience
KW - Scientific collaboratory
U2 - 10.1145/772047.772051
DO - 10.1145/772047.772051
M3 - Journal article
AN - SCOPUS:0042432155
SN - 1073-0516
VL - 10
SP - 150
EP - 176
JO - ACM Transactions on Computer-Human Interaction
JF - ACM Transactions on Computer-Human Interaction
IS - 2
ER -