Joe Corneli merge  about 9 years ago

Commit id: 860ef2865d819787f69e8e1b7bdf6dfbca5c5e9f

deletions | additions      

       

analyses current evaluation procedures used in computational  creativity, and provides a much-needed set of customisable evaluation  guidelines, the \emph{Standardised Procedure for Evaluating Creative  Systems} (SPECS) \cite{jordanous:12}. The three step process of SPECS requires the evaluator to define the concept(s) they are evaluating the system on (originally SPECS was designed to evaluate the concept of creativity). This definition is then converted into testable standards that can be used to evaluate individual systems, or comparatively evaluate multiple systems.  We propose give  a slightly modified version of her earlier evaluation guidelines, in that rather than attempt a definition and evaluation of  {\em creativity}, we follow the three steps for \emph{serendipity}. \newpage 

\begin{quote} {\em Identify a definition of serendipity that your  system should satisfy to be considered serendipitous.}\end{quote}  \noindent As above. Our computational definition of serendipity is as given in Section \ref{sec:our-model}.  %% This situation can be pictured schematically as follows:         

\begin{quote} {\em Using Step 1, clearly state what standards you use to evaluate the serendipity of your  system. }\end{quote}  \noindent Here we need to identify testable standards from our definition of computational serendipity. in other words, we now state the key parts of our definition in a form that can be evaluated as to what degree they are or are not met.  With our definition in mind, we propose the following standards for evaluating serendipity in computational systems: %% Serendipity relies on a reassessment or reevaluation -- a \emph{focus shift} in which something that was previously uninteresting, of neutral, or even negative value, becomes interesting. 

evaluated as useful, by the system and/or by an external source.}  \item[\emph{(\textbf{B - Dimensions})}] \emph{Serendipity, and its  various dimensions, can be present to a greater or lesser degree.  If the criteria above have been met, we consider the system (and optionally,  generate ratings as estimated probabilities probabilities)  along several dimensions: %  \emph{($\mathbf{a}$ - \textbf{chance})} how likely was this trigger to appear to  the system? 

\medskip  \emph{Then aggregating  $\mathbf{a}\times\mathbf{b}\times\mathbf{c}$ gives a likelihood score: low likelihood $\mathbf{a}\times\mathbf{b}\times\mathbf{c}$  and high value is $\mathbf{d}$ are  the criterion criteria  we use to say that the event was ``highly serendipitous.''} \item[\emph{(\textbf{C - Factors})}] \emph{Finally, if the criteria  from Part A are met, and if the event is deemed ``highly         

%% http://bibdesk.sourceforge.net/  %% Created for Anna Jordanous at 2015-03-26 11:11:15 12:09:00  +0000 %% Saved with string encoding Unicode (UTF-8)   @inproceedings{jordanous10,  Address = {Lisbon, Portugal},  Annote = {GA system using Ritchie's criteria as a fitness function for creativity},  Author = {Jordanous, Anna},  Booktitle = {Proceedings of the International Conference on Computational Creativity},  Date-Added = {2015-03-26 12:08:54 +0000},  Date-Modified = {2015-03-26 12:08:59 +0000},  Pages = {223-227},  Read = {1},  Title = {A Fitness Function for Creativity in Jazz Improvisation and Beyond},  Year = {2010}}  @inproceedings{jordanous2010,  Address = {Lisbon, Portugal},  Annote = {GA system using Ritchie's criteria as a fitness function for creativity},         

\subsection{Case Studies: Prior art}  \paragraph{{[}To add: Jazz.{]}} \paragraph{Evolutionary music improvisation systems.}  \citeA{jordanous10} reported a computational jazz improvisation system using genetic algorithms. Genetic algorithms, and evolutionary computing more generally, could encourage computational serendipity. We examine Jordanous's system (later given the name {\em GAmprovising} \cite{jordanous:12}) as a case study for evolutionary computing in the context of our model of computational serendipity: to what extent does GAmprovising model serendipity?  % \paragraph{{[}To add: HR.{]}} 

% As a general comment, we would say that this is largely how  % \emph{research and development} of recommender systems works, but  % without the same levels of system automony envisioned here.  \small  \begin{table}[ht]%dp]  \caption{Summary of case studies as viewed with our computational serendipity model}  \begin{center}  \begin{tabular}{|c|l|l|}  \hline  {\textbf Model part} & Evolutionary music systems & Recommender systems \\  \hline  %{\em Key Condition} && \\  %* Focus shift && \\  %\hline  {\em Components} && \\  * Serendipity trigger && Input from user behaviour \\  * Prepared mind & & Through user model/domain model \\  * Bridge & & Elements identified outside clusters \\  * Result & & Dependent on organisation goals \\  \hline  {\em Dimensions} && \\  * Chance & & If learning from user behaviour \\  * Curiosity & & Making unusual recommendations \\  * Sagacity & & Updating models from user behaviour \\  * Value & & As per business metrics/objectives \\  \hline  {\em Environmental} && \\  {\em Factors} && \\  * Dynamic world && As precondition for testing system's \\  world && \hspace{3mm} influences on user behaviour\\  * Multiple && User model and domain model\\  contexts && \\  * Multiple && Making recommendations, learning\\  * tasks && \hspace{3mm}from users, updating models \\  * Multiple && Experimental design, psychology, \\  influences && \hspace{3mm} domain understanding\\  \hline  \end{tabular}  \end{center}  \label{caseStudies}  \end{table}%  \normalsize      Binary files a/figures/model-diagram/mm-model-diagram-no-desc.png and b/figures/model-diagram/mm-model-diagram-no-desc.png differ        

    MODIFIED="1427370123836" MODIFIED="1427374772645"  TEXT="Computational model of serendipity">     MODIFIED="1427370123738" MODIFIED="1427374771070"  POSITION="right" TEXT="Phases">     MODIFIED="1427370163177" MODIFIED="1427374741959"  TEXT="1. Discovery">             MODIFIED="1427370163864" MODIFIED="1427374741470"  TEXT="2. Invention">      

 
 
  MODIFIED="1427370123760" MODIFIED="1427374740871"  POSITION="right" TEXT="Key condition">     MODIFIED="1427370164688" MODIFIED="1427374742607"  TEXT="Focus shift">      

 
 
  CREATED="1427369178393" ID="ID_1382365722" MODIFIED="1427374744353" POSITION="right" TEXT="Components">                 
                                                CREATED="1427368709710" ID="ID_563045342" MODIFIED="1427370123773" MODIFIED="1427374747155"  POSITION="right">      

      MODIFIED="1427370165529" MODIFIED="1427374751573"  TEXT="Chance">             MODIFIED="1427370167384" MODIFIED="1427374748460"  TEXT="Curiosity">      

   
  MODIFIED="1427370166712" MODIFIED="1427374752829"  TEXT="Sagacity">             MODIFIED="1427370168032" MODIFIED="1427374753509"  TEXT="Value">      

 
    MODIFIED="1427370123785" MODIFIED="1427374755052"  POSITION="right" TEXT="Environmental Factors">     MODIFIED="1427370168816" MODIFIED="1427374756549"  TEXT="Dynamic World">      

   
  MODIFIED="1427370169561" MODIFIED="1427374757276"  TEXT="Multiple contexts">             MODIFIED="1427370170585" MODIFIED="1427374766079"  TEXT="Multiple tasks">             MODIFIED="1427370172033" MODIFIED="1427374766709"  TEXT="Multiple influences">      

 
 
                                                                   
      Binary files a/figures/model-diagram/mm-model-diagram.png and b/figures/model-diagram/mm-model-diagram.png differ