Lucas Fidon edited subsection_Mutual_Information_definition_and__.tex  almost 8 years ago

Commit id: 2e28131094c728edeaea9845acd9b5b9ce26d594

deletions | additions      

       

  Let $X: P \rightarrow E$ be a random variable with $E$ a discrete probability space.  The entropy of $X$ is defined as:  $ \[  S(X) = \sum_{x in E}P_{X}(x)*log(P_{X}(x))$ E}P_{X}(x)*log(P_{X}(x)) \]    The entropy has three interpretations:   \begin{itemize}