Evaluation
\label{evaluation}
Formative evidence for the value of the tasks and designs listed above
was provided via multiple methods over many months and will not be
discussed here due to their diversity. Our scientific goals are to
result in the exploration of more ideas, improve awareness of activity
in the field, and spark new interactions between laboratories. Prior to
launch we conducted a closed beta with 15 users, 10 of whom represented
our target users with five others in diverse roles that could help us
explore ancillary value. The goals of beta were to find errors in the
data and interactions, prepare the team for launch logistics, and begin
to measure value. We describe results below for the target 10 users.
Beta users were introduced to the system in group sessions that included
a demo along with participation instructions. Before beta, users filled
out a baseline survey with Likert-style questions identical to a
capstone survey taken at the end in order to measure the influence of
the DataSpace. Results on the 5-point response scale were not normally
distributed, so differences were tested using a paired, 1-tailed
Wilcoxon Signed-Rank test with significance at p ≤ .05. The capstone
survey included additional sections on DataSpace usage and the System
Usability Scale (SUS) [16].
Users accessed the DataSpace through the Web on their own devices and at
their convenience. In the first four weeks users were assigned specific
scientific questions that ensured coverage of major application areas
and dealt with key features for feedback, such as those described in the
Application section. In the next four weeks users were asked to choose
their own tasks. In the final one-four weeks before launch they were
free to use the system or not. We collected feedback through a group
session after one week of use, occasional individual interviews, Google
Analytics, session playback using Inspectlet, and weekly surveys.