Tasks
\label{tasks}
While VA has always discussed applications in science, it began in
intelligence, law enforcement, and emergency response [13]. It is
telling that the visual analytics acronym VACCINE is actually a
Department of Homeland Security project on command and control. Still,
basic analysis processes are the same. The foraging and sense-making
loops, the process of explore, enrich, and exploit, and 9 out of 10 of
Amar et al.’s low-level components of analytic tasks apply in our
qualitative observations [14][13][15]. Some important
differences are worth a brief discussion. (1) Traditional analysts may
get VA tool support for creating and assessing confidence in reasoning
chains based on evidence with mixed reliability, while vaccine
investigators ultimately rely on statistics. (2) Analysts often make
written reports to decision makers, while outcomes for vaccine
investigators may be less direct. A new collaboration, influence on a
new study design or vaccine concept, or a new meta-analysis may be many
steps removed from hypothesis generation. (3) Analysts work with
independently produced data, often in teams. Vaccine investigators are
used to working with data they produced themselves and which they may
feel belongs to them. They choose trusted collaborations carefully to
preserve their reputation and the novelty and integrity of their work.
(4) Perhaps most importantly, vaccine investigators are not full-time
analysts. Working with data across studies and assays on self-service
interactive timescales is fundamentally new. As a result of such
differences, our early research led us away from direct collaboration
inside the tool, prompted us to add data contributors’ contact
information and easy export for outside impact, and suggested that broad
adoption required simplification, guidance, and smart default choices
rather than paired analysis with a tool expert.
Scenario collection and iterative prototyping defined and prioritized
our key tasks. While the concept of the DataSpace was new, investigators
easily thought of questions to ask of such a system. Publications and
staff who field requests for unplanned or meta-analysis were additional
sources of questions. Our goal is to enable any investigator to conduct
these tasks quickly and on their own:
-
Find groups of subjects across studies based on multiple attributes
that may be categorical or quantitative, and may vary over time or
not. Filter data accordingly.
-
Look for a pattern in a measure over time for a group or individual,
even across studies.
-
Look for group differences based on properties of the subjects or
their assay measures.
-
Look for a pattern between two assays at all available time points or
a subset of time points.
-
Find detailed metadata about studies and assays as an end in itself,
to influence the interpretation of a visualization, or to influence
which data to use.
-
Export data for further analysis.
Early on we found that even the most expert investigators are not used
to some of the novel interpretation problems of combining complex data.
Prototyping and task walkthroughs led to several key application
principles:
-
Top-down. Filters pare down the entire data space rather than building
up a set for analysis in order to make clear what is available and
enable serendipitous discovery.
-
Breadth. Filter subjects but preserve dimensions (all available data
about those subjects) to prioritize cross-study and cross-assay
comparison.
-
Context. Expose the relationship of the filtered data to the whole to
clarify the unexpected impacts of filtering.
-
Uncertainty. Make users aware of data that meet their filters but
cannot be plotted in the current view. Avoid statistics that would
imply meaning in views that may be spurious or confounded by unseen
factors.
-
Guidance. For broadest value, design as if users are not familiar with
assay and study details.