Authorea

Joe Corneli revise recommendations / challenges section about 9 years ago

Commit id: c4e2a045c359b9534734fb76b1a3f0fb3025ee9a

deletions | additions

%% Saved with string encoding Unicode (UTF-8) @article{d2012creativity, title={Creativity through autonomy and interaction}, author={d’Inverno, Mark and Luck, Michael}, journal={Cognitive Computation}, volume={4}, number={3}, pages={332--346}, year={2012}, publisher={Springer} } @article{al2012creativity, title={Creativity and autonomy in swarm intelligence systems}, author={al-Rifaie, Mohammad Majid and Bishop, John Mark and Caines, Suzanne}, journal={Cognitive Computation}, volume={4}, number={3}, pages={320--331}, year={2012}, publisher={Springer} } @article{saunders2012towards, title={Towards autonomous creative systems: A computational approach}, author={Saunders, Rob}, journal={Cognitive Computation}, volume={4}, number={3}, pages={216--225}, year={2012}, publisher={Springer} } @article{ritchie07, Annote = {Introduction Assumptions - What Kinds of Activity are Creative? - The Basis in Human Creativity - Sources of Evidence [ignoring process - but is this justified?] - What Kind of Program? - P-Creativity and H-Creativity - Essential properties * Novelty * Quality - ALSO - * Typicality The Framework - Basic Items - Rating the Output - The Objects Generated - Inspiring Set, Program and Results Evidence of Creativity - Preliminaries - The Criteria Related Proposals - Fine Tuning - Other Formalisations * PWC: General Extensions, More Specific Proposals, PWC-Overall * Koza et al's Guidelines Applications of the Criteria - A Poetry Generator [WASP, Gervas 2002] - A Concept Generator [Divago, Pereira 2005] - A Paraphrase Generator [Dupond, (assessed in) Pereira 2005, (reported in) Mendes et al 2004] - Pereira et al.'s Summing Up - A Melody Generator Discussion - Use of the criteria [by others] - The meaning of the criteria Possible extensions - Similarity - The Contribution of the Designer - Self-Rating of Output - Multiple Runs - Random Generation Conclusion Q. e.g. simulation using Ritchie's criteria as GA - how do you find appropriate values for threshold levels and parameters, without overfitting to your system? This is left unsaid by Ritchie except for a small section in p. 93 Q. reliance on inspiring set - does this mean that Ritchie's criteria are heavily influenced by a case-based reasoning type approach? e.g. criteria 9 checks that the examples in the inspiring set can be replicated - what if this isn't important in your system (for example... ***) or if there is no inspiring set? (*** check feedback from ICCCX paper as I think this is discussed there ***) Q. In fact is the whole approach too closely fitted to a computational approach such as used by Ritchie in JAPE or STANDUP, at the expense of applying more generally to a variety of approaches? For debate about computer creativity: Dreyfus 1979, Lovelace, Weizenbaum 1976 Computer power and human reason It is outside the scope of this paper to build a model of creativity or of the creative process, or to define what is neeeded to show evidence of creativity (p. 68). The aim is to formally define observable factors which possibly relate to creativity.},

% As a general comment, we would say that this is largely how % \emph{research and development} of recommender systems works, but % without the same levels of system automony envisioned here. \begin{table}[ht]%dp] \caption{Summary: applying computational serendipity model to positive case studies} \begin{center} \begin{table}[ht!] {\centering \renewcommand{\arraystretch}{1.5} \footnotesize \begin{tabular}{|c|l|l|} \hline \begin{tabular}{p{.7in}@{\hspace{.1in}}p{1.9in}@{\hspace{.1in}}p{1.9in}} \multicolumn{1}{c}{} & Evolutionary \multicolumn{1}{c}{\textbf{Evolutionary music systems systems}} & Recommender systems \\ \hline \hline %{\em Key Condition} && \\ %Focus shift && \\ %\hline {\em \multicolumn{1}{c}{\textbf{Recommender systems}} \\[-.1in] \multicolumn{1}{l}{\em Components} && & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\ \hline \hline Serendipity trigger \cline{2-3} \textbf{Serendipity trigger} & Evolutionary operations and user input & Input from user behaviour \\ \hline Prepared mind % \cline{2-3} \textbf{Prepared mind} & Musical knowledge, evolution mechanisms & Through user model/domain user/domain model \\ \hline Bridge % \cline{2-3} \textbf{Bridge} & Creation of newly-evolved Improvisors & Elements identified outside clusters \\ \hline Result % \cline{2-3} \textbf{Result} & Music generated by fittest Improvisors& Dependent on organisation goals \\ \hline \hline {\em \cline{2-3} \multicolumn{1}{l}{\em Dimensions} && & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\ \hline \hline Chance \cline{2-3} \textbf{Chance} & If discovered in huge search space & If learning from user behaviour \\ \hline Curiosity % \cline{2-3} \textbf{Curiosity} & If a particular user notes an Improvisor & Making unusual recommendations \\ \hline Sagacity % \cline{2-3} \textbf{Sagacity} & User appreciation of Improvisor over time & Updating models after user behaviour \\ \hline Value % \cline{2-3} \textbf{Value} & Via fitness function (proxy of creativity) & As per business metrics/objectives \\ \hline \hline {\em Environmental} && \\ {\em \cline{2-3} \multicolumn{1}{l}{\em Factors} && & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\ \hline \hline Dynamic \cline{2-3} \textbf{Dynamic world} & Continuous computational evolution& evolution and changes in user tastes& As precondition for testing system's\\ world & \hspace{3mm}and changes in user tastes & \hspace{3mm} influences on user behaviour\\ \hline Multiple %\cline{2-3} \textbf{Multiple contexts} & Multiple users opinions? & User model and domain model\\ contexts & \hspace{3mm}users opinions? & \\ \hline Multiple % \cline{2-3} \textbf{Multiple tasks} & Evolving Improvisors, generating music, & Making recommendations, learning\\ tasks & \hspace{3mm}collecting collecting user input, fitness calculations& \hspace{3mm}from calculations & Making recommendations, learning from users, updating models \\ \hline Multiple % \cline{2-3} \textbf{Multiple influences} & Through various musical& musical parameter combinations& Experimental design, psychology,\\ influences &\hspace{3mm}parameter combinations& \hspace{3mm} domain understanding\\ \hline \cline{2-3} \end{tabular} \end{center} \label{caseStudies} \par} \bigskip \caption{Summary: applying computational serendipity model to positive case studies\label{caseStudies}} \end{table}% \normalsize

Spellchecker programs could indeed be said to have a \textbf{prepared mind}, in that they are constructed with internal dictionaries with which to check spelling and ways of calculating what a misspelled word might be. Given our above discussion of how the system might be serendipitous, the \textbf{serendipity trigger} could be seen as the user misspelling a word and the system suggesting a correction (which might cause the user to think of alternative possibilities for their text that they had not previously conceived). The \textbf{result} of this serendipity would be new text typed in by the user in response to the serendipity trigger, but the \textbf{bridge} from trigger to result would have been built by the user, not the system. So all components in our model are not present in spellchecker software, as the ability to create a bridge from trigger to result is absent in the system. At this point we do not need to look further at the dimensions, or attributes of these components, because not all components are present in spellchecker software. % \paragraph{{[}To add: HR.{]}} *** AJ PERHAPS WE DON'T NEED TO?

\emph{autonomy}, \emph{learning}, \emph{sociality}, and \emph{embedded evaluation and ethics}. The argument presented here is not supported by experimental results, but focuses on In the current work have limited ourselves to clarifying conceptual issues and examining design implications. % We indicate several possible further directions for implementation work in each of our case studies. We have also drawn attention to more theoretical questions related to doing program design for an autonomous programming context. Our examples show that serendipity is not foreign to computing practice. There are further gains to be had for research in computing by planning -- and programming -- for

\subsection{Recommendations} \label{sec:recommendations} Our thought experiment in Section \ref{sec:ww} develops a design illustrating the relationship between creativity at the level of artefacts (e.g.~new poems) and creativity at the level of \emph{problem specification}. The search for connections that make raw data into ``strategic data'' is an appropriate theme for researchers in computational creativity to grapple with. \citeA{stakeholder-groups-bookchapter} outlined a general programme \subsection{Challenges for computational creativity, and examined perceptions of creativity in computational systems found among members of the general public, Computational Creativity researchers, and creative communities -- understood as human communities. We should now add a fourth important ``stakeholder'' group in computational creativity research: computer systems themselves. Creativity may look very different to this fourth stakeholder group than it looks to us. We should help computers evaluate their own results and creative process. As Campbell \citeyear{campbell} writes: ``serendipity presupposes a smart mind.'' We may be aided in our pursuit by recalling Turing's proposal that computers should ``be able to converse with each other to sharpen their wits'' \cite{turing-intelligent}. Other fields, including computer Chess, Go, and argumentation have achieved this, and to good effect. The Writers Workshop described in Section \ref{sec:ww} is an example of one such social model, but more fundamentally, it is an example of \emph{learning from experience}. The Workshop model ``personifies'' the wider world in the form of one or several critics. It is clearly also possible for a lone creative agent to take its own critical approach in relationship to the world at large, using an experimental approach to generate feedback, and then looking for models to fit this feedback. future research} \label{sec:recommendations} To summarise: we advance the following further criteria for research in computational serendipity, viewing the concepts in Section \ref{sec:by-example} through the practice scenarios we have discussed. % \subsubsection*{The ``serendipity programme'' for computational creativity} \begin{itemize} \item \textbf{Autonomy}: Our thought experiment in Section \ref{sec:ww} develops a design illustrating the relationship between creativity at the level of artefacts (e.g.~new poems) and creativity at the level of \emph{problem specification} (capturing new poetic concepts). The search for connections that make raw data into ``strategic data'' is an appropriate theme for researchers in computational creativity to grapple with. In the standard cybernetic model, we control computers, and we also control the computer's operating context. There is little room for serendipity because there is nothing outside of our direct control. Von Foerster \citeyear[p. 286]{von2003cybernetics} advocated a \emph{second-order cybernetics} in which ``the observer who enters the system shall be allowed to stipulate his own purpose.'' An eventual corollary of \emph{A primary challenge in the serendipitous operation of computers will be that \emph{Computational for computational agents can to specify their own problems to work on.} \end{itemize} \begin{itemize} \item \textbf{Learning}: If we admit The Writers Workshop described in Section \ref{sec:ww} is fundamentally an example of a design for a system that can \emph{learn from experience}. The Workshop model ``personifies'' the wider world in the possibility form of one or several critics. It is clearly also possible for a lone creative agent to take its own critical approach in relationship to the world at large, using an experimental approach to generate feedback, and then looking for models to fit this feedback. We are led to consider computational agents that operate our world rather than a circumscribed microdomain, and that are curious about this world, then another corollary world. \emph{A second challenge is that \emph{Computational for computational agents will to learn more and more about the world we live in.} \end{itemize} \begin{itemize} \item \textbf{Sociality}: As Campbell \citeyear{campbell} writes: ``serendipity presupposes a smart mind.'' We may be aided in our pursuit by recalling Turing's proposal that computers should ``be able to converse with each other to sharpen their wits'' \cite{turing-intelligent}. Other fields, including computer Chess, Go, and argumentation have achieved this, and to good effect. Deleuze \citeyear[p. 26]{deleuze1994difference} wrote: ``We learn nothing from those who say: `Do as I do'. Our only teachers are those who tell us to `do with me'[.]'' Turing recognised that computers would have to be coached in the direction of social learning, but that once they attain that standard they will learn much more quickly. A \emph{A third corollary of serendipitous computing challenge is that \emph{Computational for computational agents will to interact in a recognisably social way with us and with each other, resulting in emergent effects.} \end{itemize} \begin{itemize} \item \textbf{Embedded evaluation evaluation}: \citeA{stakeholder-groups-bookchapter} outlined a general programme for computational creativity, and ethics}: Finally, examined perceptions of creativity in computational systems found among members of the general public, Computational Creativity researchers, and creative communities -- understood as human communities. We should now add a fourth corollary important ``stakeholder'' group in computational creativity research: computer systems themselves. Creativity may look very different to this fourth stakeholder group than it looks to us. We should help computers evaluate their own results and creative process. \emph{A fourth challenge is that \emph{Computational for computational agents will to evaluate their own creative process and products.}It is our responsibility as system designers to teach them how to make evaluations in an ethical manner. This is exemplified by the preference for a ``non-zero sum'' criterion for value suggested in our discussion of the dimensions of serendipity in Section \ref{sec:by-example}. \end{itemize} It is our responsibility as system designers to teach them how to make evaluations in both a reasonable and an ethical manner. This is exemplified by the preference for a ``non-zero sum'' criterion for value suggested in our discussion of the dimensions of serendipity in Section \ref{sec:by-example}. A quick survey of word occurrences from the last three years a recent special issue of proceedings from the International Conference on Computational Creativity \emph{Cognitive Computation} shows that related themes are broadly active in the research community. community.\footnote{Articles converted to text via {\tt pdftotext -layout}, individual counts found via {\tt tr \textquotesingle~\textquotesingle~\textquotesingle\textbackslash n\textquotesingle~< \$i | grep -c "stem*"}, and total word counts via {\tt wc -w}. The corresponding counts for the \emph{current} paper are 9, \emph{25}, \emph{15}, \emph{43} and 11.9K.} Here \emph{italics} indicates that the word stem accounted for .1\% of the article or more, and \textbf{\emph{bold italics}} indicates that it accounted for 1\% or more. \medskip {\centering \setlength{\tabcolsep}{7pt} \small \hspace{-7pt}\begin{tabular}{r|l|l|l|l|l|l|} \multicolumn{1}{r}{} & \multicolumn{1}{c}{autonom*} & \multicolumn{1}{c}{learn*} & \multicolumn{1}{c}{social*} & \multicolumn{1}{c}{evaluat*} & \multicolumn{1}{c}{ethic*} & \multicolumn{1}{c}{\textbf{total}}\\ \cline{2-7} 2012&36 (.1\%) & 124 (1\%) & 74 (.6\%) & 478 (3.8\%) & 3 (<.1\%) & 12472 \setlength{\tabcolsep}{5pt} \tiny \begin{tabular}{ccccccccccccccc} paper \# &1 &2 &3 &4 &5 &6 &7 &8 &9 &10 &11 &12 &13 &14 \\ \cline{2-15} "autonom.*" &0 &\textbf{\emph{32}} &\emph{12} &\emph{41} &0 &1 &\emph{31} &2 &1 &\emph{92} &11 &2 &5 &\textbf{\emph{22}} \\ "learn.*" &6 &2 &2 &\emph{14} &\emph{9} &\textbf{\emph{118}} &\emph{14} &\emph{18} &\emph{44} &\emph{12} &11 &\emph{42} &\emph{44} &2 \\ "social.*" &0 &0 &\emph{23} &\emph{25} &0 &1 &2 &\emph{10} &\emph{19} &\emph{19} &8 &\emph{21} &13 &2 \\ "evaluat.*" &0 &1 &\emph{11} &\emph{20} &0 &1 &3 &6 &4 &9 &8 &2 &\textbf{\emph{304}} &0 \\ \cline{2-15} \textbf{total(K)} & 8.3 % &8337 (/ 6 8337.0) & 2.2 % &2221 (/ 32 2221.0) 0.0135074290859973 & 7.5 % &7507 (/ 12 7507.0) 0.0015985080591447982 (/ 23 7507.0) 0.0026641800985746636 (/ 11 7507.0)0.001465299054216065 & 7.4 % &7453 (/ 41 7453.0) 0.004964443848114853 (/ 14 7453.0) 0.0009392191064001073 (/ 16 7453.0) 0.002146786528914531 (/ 19 7453.0) 0.0025493090030860054 & 8.6 % &8675 (/ 9 8675.0) & 5.8 % & 5816 (/ 89 5816.0) 0.015302613480055021 &10.3 % &10341 (/ 30 10341.0) 0.002901073397156948 & 9.6 % &9632 (/ 18 9632.0) 0.0018687707641196014 (/ 10 9632.0)0.0010382059800664453 &10.8 % &10851 (/ 36 10851.0) 0.0033176665745092617 &11.6 % &11693 (/ 92 11693.0)0.007867955186863935 (/ 12 11693.0) &14.4 % &14407 (/ 11 14407.0) 0.0007635177344346498 &10.8 % &10840 (/ 31 10840.0) 0.0028597785977859777 &25.3 % &25326 (/ 13 25326.0) (/ 44 25326.0) 0.0008291873963515755 (/ 304 25326.0) 0.011174287293690278 & 1.6 % &1673 (/ 21 1673.0) 0.012552301255230125 \\\cline{2-7} 2013&69 (.6\%) & 218 (1.7\%) & 116 (.9\%) & 360 (2.9\%) & 0 (0\%) & 12531\\ \cline{2-7} 2014&77 (.4\%) & 292 (1.6\%) & 360 (2\%) & 993 (5.4\%) & 2 (<.1\%) & 18192\\ \cline{2-7} \end{tabular} \par } \bigskip \noindent Given Paper 4, Rob Saunder's \citeyear{saunders2012towards} ``Towards Autonomous Creative Systems: A Computational Approach'' was the community's active interest in evaluation, only one to emphasise all four of our themes. Saunders asks: ``What would it mean to produce an autonomous creative system? How might we believe that the proposed standards approach this task? And, how would we have outlined know if we had succeeded?'' He argues for an approach ``that models personal motivations, social interactions and the evolution of domains.'' Paper 10, d'Inverno and Luck's \citeyear{d2012creativity} ``Creativity Through Autonomy and piloted Interaction'', also contains a theoretical engagement with these themes, and presents a formalism for evaluating computational serendipity will multi-agent systems that could usefully be of use. adapted to model serendipitous encounters. We hope believe that our clarifications to the often subtle multifaceted concept of serendipity will help to encourage future computer-aided (and computer-driven) investigations of the constituent processes of above themes and their interrelationships. We are particularly interested in the relationship between discovery and invention. invention, and we discuss some of our own plans in this direction below.

\usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} \usepackage{textcomp} \usepackage{setspace} \usepackage[framemethod=tikz]{mdframed}

\newcommand{\keywords}[1]{\par\addvspace\baselineskip \noindent\keywordname\enspace\ignorespaces#1} \usepackage{array,booktabs} \newcommand{\tcap}[1]{ \begin{minipage}{.7in} {\centering #1 \par} \end{minipage} } \newcommand{\inlineitem}[1]{% \begingroup% \setlength\itemindent{-1em}%