this is for holding javascript data
Anna Jordanous modifying the text on SPECS to cover different layers of evaluation (successfully? any comments?)
about 9 years ago
Commit id: 9945ad62247af2d4f584e45574987be079fe72fb
deletions | additions
diff --git a/SPECS-begins.tex b/SPECS-begins.tex
index 2c1b2bb..a3c05bd 100644
--- a/SPECS-begins.tex
+++ b/SPECS-begins.tex
...
analyses current evaluation procedures used in computational
creativity, and provides a much-needed set of customisable evaluation
guidelines, the \emph{Standardised Procedure for Evaluating Creative
Systems} (SPECS) \cite{jordanous:12}.
The three step process of SPECS requires the evaluator to define the concept(s) they are evaluating the system on (originally SPECS was designed to evaluate the concept of creativity). This definition is then converted into testable standards that can be used to evaluate individual systems, or comparatively evaluate multiple systems.
We
propose give a slightly modified version of her earlier evaluation
guidelines, in that rather than attempt a definition and evaluation of
{\em creativity}, we follow the three steps for \emph{serendipity}.
...
\begin{quote} {\em Identify a definition of serendipity that your
system should satisfy to be considered serendipitous.}\end{quote}
\noindent
As above. Our computational definition of serendipity is as given in Section \ref{sec:our-model}.
%% This situation can be pictured schematically as follows:
diff --git a/SPECS-continues.tex b/SPECS-continues.tex
index aa96fef..d321604 100644
--- a/SPECS-continues.tex
+++ b/SPECS-continues.tex
...
\begin{quote} {\em Using Step 1, clearly state what standards you use to evaluate the serendipity of your
system. }\end{quote}
\noindent
Here we need to identify testable standards from our definition of computational serendipity. in other words, we now state the key parts of our definition in a form that can be evaluated as to what degree they are or are not met.
With our definition in mind, we propose the following standards for evaluating serendipity in computational systems:
%% Serendipity relies on a reassessment or reevaluation -- a \emph{focus shift} in which something that was previously uninteresting, of neutral, or even negative value, becomes interesting.
...
evaluated as useful, by the system and/or by an external source.}
\item[\emph{(\textbf{B - Dimensions})}] \emph{Serendipity, and its
various dimensions, can be present to a greater or lesser degree.
If the criteria above have been met, we
consider the system (and optionally, generate ratings as
estimated
probabilities probabilities) along several dimensions:
%
\emph{($\mathbf{a}$ - \textbf{chance})} how likely was this trigger to appear to
the system?
...
\medskip
\emph{Then
aggregating $\mathbf{a}\times\mathbf{b}\times\mathbf{c}$ gives a
likelihood score: low likelihood
$\mathbf{a}\times\mathbf{b}\times\mathbf{c}$ and high value
is $\mathbf{d}$ are the
criterion criteria we use to say that the event was ``highly serendipitous.''}
\item[\emph{(\textbf{C - Factors})}] \emph{Finally, if the criteria
from Part A are met, and if the event is deemed ``highly