Independence of Random Variables

Ponder...

Discrete random variables with values in are independent when

for any : joint masses factorise into products of marginal masses.

Definition 3.6 (Independence)

Let be discrete random variables with values in . They are independent if for any ,

Subcollections are independent

Proposition 3.25

If are independent, then is independent of .

Proof of subcollection independence

Suppose are independent. To show is independent of , check that for all ,

The same argument applies to any subcollection.

Expectations factorise

Lemma 3.26

Let and be 2 independent random variables, and . Then

Proof of factorisation

Let and be independent random variables and . With and ,

Independence enters at the third line, where the joint mass factorises.

Factorisation criterion for densities

Theorem 4.23

Let be a random vector with density .

Suppose are independent with densities , then for all ,
Conversely, suppose factorises as for some non-negative functions on . Then are independent with densities proportional to .

Proof of the density factorisation criterion

Suppose are independent with densities . Then

so the joint distribution function is that of a random vector with density .

Conversely, suppose factorises as for non-negative functions . Then

where the denominators normalise each factor since . The joint distribution function is a product of functions of single coordinates, so are independent with densities proportional to .

Stated in

Definition 3.6 (Independence)§3 Discrete Random Variables
Proposition 3.25§3.2 Variance and Covariance
Lemma 3.26§3.2 Variance and Covariance
Theorem 4.23§4.6.1 Introduction