For any 2 random variables $X$, $Y$:
$
\begin{align}
I(X;Y) &= H(X) - H(X|Y) \\
&= D[p(x,y)||p(x)p(y)] \\
&= D[p(x|y)||p(x)] \\
&= D[p(y|x)||p*y)]
\end{align}
$
## References
---
- Links: [[KL-Distribution divergence|KL-Divergence]] [[Cross-entropy]]
- Created at: 2023-08-05