Evaluation

\label{evaluation}
Formative evidence for the value of the tasks and designs listed above was provided via multiple methods over many months and will not be discussed here due to their diversity. Our scientific goals are to result in the exploration of more ideas, improve awareness of activity in the field, and spark new interactions between laboratories. Prior to launch we conducted a closed beta with 15 users, 10 of whom represented our target users with five others in diverse roles that could help us explore ancillary value. The goals of beta were to find errors in the data and interactions, prepare the team for launch logistics, and begin to measure value. We describe results below for the target 10 users.
Beta users were introduced to the system in group sessions that included a demo along with participation instructions. Before beta, users filled out a baseline survey with Likert-style questions identical to a capstone survey taken at the end in order to measure the influence of the DataSpace. Results on the 5-point response scale were not normally distributed, so differences were tested using a paired, 1-tailed Wilcoxon Signed-Rank test with significance at p ≤ .05. The capstone survey included additional sections on DataSpace usage and the System Usability Scale (SUS) [16].
Users accessed the DataSpace through the Web on their own devices and at their convenience. In the first four weeks users were assigned specific scientific questions that ensured coverage of major application areas and dealt with key features for feedback, such as those described in the Application section. In the next four weeks users were asked to choose their own tasks. In the final one-four weeks before launch they were free to use the system or not. We collected feedback through a group session after one week of use, occasional individual interviews, Google Analytics, session playback using Inspectlet, and weekly surveys.