Update.

[tex.git] / inftheory.tex

diff --git a/inftheory.tex b/inftheory.tex
index 33ccfe5..eb9e988 100644 (file)
--- a/inftheory.tex
+++ b/inftheory.tex
@@ -116,7 +116,7 @@ that quantifies the amount of information shared by the two variables.
 
 \section{Conditional Entropy}
 
-Okay given the visible interest for the topic, an addendum: Conditional entropy is the average of the entropy of the conditional distribution:
+Conditional entropy is the average of the entropy of the conditional distribution:
 %
 \begin{align*}
 &H(X \mid Y)\\
@@ -126,7 +126,9 @@ Okay given the visible interest for the topic, an addendum: Conditional entropy
 
 Intuitively it is the [minimum average] number of bits required to describe X given that Y is known.
 
-So in particular, if X and Y are independent 
+So in particular, if X and Y are independent, getting the value of $Y$
+does not help at all, so you still have to send all the bits for $X$,
+hence
 %
 \[
   H(X \mid Y)=H(X)
@@ -138,7 +140,7 @@ if X is a deterministic function of Y then
   H(X \mid Y)=0
 \]
 
-And since if you send the bits for Y and then the bits to describe X given that X is known you have sent (X, Y), we have the chain rule:
+And since if you send the bits for Y and then the bits to describe X given that Y, you have sent (X, Y), we have the chain rule:
 %
 \[
 H(X, Y) = H(Y) + H(X \mid Y).