this is for holding javascript data
Sven Schmit edited evaluation.tex
over 9 years ago
Commit id: bdb7e82f27570d9d1794d5082e557a91ae55ae1a
deletions | additions
diff --git a/evaluation.tex b/evaluation.tex
index b69c232..c76ee58 100644
--- a/evaluation.tex
+++ b/evaluation.tex
...
\section{Evaluation}
Rather than comparing a proposed method to a baseline and an oracle, we propose to investigate two models as explained above, and compare their performance.
This has two reasons.
First, we think comparing these two methods is interesting: are agents able to cooperate without communication simply because rewards are aligned, or does a central `commander' improve efficiently dramatically?
Second, it is non-trivial to come up with baseline and oracle methods for these tasks.
There are two specific metrics we can use to determine performance
\begin{itemize}
\item Time till task completion,
\item Fraction of success in $T$ seconds.
\end{itemize}
Besides using these quantitative measures, the animations produced can give us a very clear picture of what is going on and whether the actions of dogs make sense.
Using both a quantitative and a qualitative approach to evaluation, we think it is possible to accurately judge the success of the learning algorithms.