Sven Schmit edited evaluation.tex  over 9 years ago

Commit id: bdb7e82f27570d9d1794d5082e557a91ae55ae1a

deletions | additions      

       

\section{Evaluation}  Rather than comparing a proposed method to a baseline and an oracle, we propose to investigate two models as explained above, and compare their performance. This has two reasons.  First, we think comparing these two methods is interesting: are agents able to cooperate without communication simply because rewards are aligned, or does a central `commander' improve efficiently dramatically?  Second, it is non-trivial to come up with baseline and oracle methods for these tasks.   There are two specific metrics we can use to determine performance  \begin{itemize}  \item Time till task completion,  \item Fraction of success in $T$ seconds.  \end{itemize}  Besides using these quantitative measures, the animations produced can give us a very clear picture of what is going on and whether the actions of dogs make sense.   Using both a quantitative and a qualitative approach to evaluation, we think it is possible to accurately judge the success of the learning algorithms.