Resume
Research
Learning
Blog
Teaching
Jokes
Kernel Papers

# Random Variables

At the heart of probability is the notion of a random variable. Intuitively, a random variable is a variable whose value depends on the outcome of some random process (e.g. a coin flip).

### Independence

Two random variables are independent if their joint probability is equal to the product of the marginal probabilities:

$P(A, B) = P(A) P(B)$

If $$A, B$$ are independent, then so are the following:

\begin{align*} P(A, B^c) &= P(A) - P(A, B)\\ &= P(A) - P(A) P(B)\\ &= P(A)(1 - P(B))\\ &= P(A) P(B^c)\\ P(A^c, B) &= P(B) - P(A, B)\\ &= P(B)(1 - P(A))\\ &= P(B)P(A^c)\\ P(A^c, B^c) &= P(A^c) - P(A^c, B^c)\\ &= P(A^c) - P(A^c)P(B)\\ &= P(A^c)(1 - P(B))\\ &= P(A^c)P(B^c) \end{align*}

# Properties

• Law of Total Expectation
$\mathbb{E}_{p(X)}[X] = \mathbb{E}_{p(Y)}[\mathbb{E}_{p(X \lvert Y)}[X]]$

Proof:

\begin{align*} \mathbb{E}_{p(X)}[X] &= \int_{X} X p(X) dX\\ &= \int_{X} X \int_{Y} p(X, Y) dX dY\\ &= \int_{Y} p(Y) \int_{X} X p(X\lvert Y) dX dY\\ &= \int_{Y} p(Y) \mathbb{E}_{p(X \lvert Y)}[X]\\ &\mathbb{E}_{p(Y)}[\mathbb{E}_{p(X \lvert Y)}[X]] \end{align*}