Kekkei - Advancing Financial Science For Everyone

Introduction

Independence is one of the most important concepts in probability. When events are independent, knowing one occurred tells us nothing about the other. This simplifies calculations and underlies many probability models.

Learning Objectives:

Master the definition of independence
Distinguish pairwise from mutual independence
Identify and test independence in practice
Understand conditional independence

Independence Definition

Events $A$ and $B$ are independent if:

$P(A \cap B) = P(A) \cdot P(B)$

Equivalent formulations (when $P(B) > 0$ ):

$P(A | B) = P(A)$ (knowing $B$ doesn't change probability of $A$ )
$P(B | A) = P(B)$ (knowing $A$ doesn't change probability of $B$ )

Intuition: Learning that $B$ occurred provides no information about $A$ .

Example 1: Coin Flips

Two fair coin flips are independent:

$P(H_1 \cap H_2) = P(H_1) \cdot P(H_2) = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}$

Knowing the first flip is heads doesn't change the probability of the second being heads.

python

import random

def test_independence_coin_flips(n_trials=10000):
    """Test independence of two coin flips"""
    count_both_heads = 0
    count_first_heads = 0
    count_second_heads = 0
    
    for _ in range(n_trials):
        flip1 = random.choice(['H', 'T'])
        flip2 = random.choice(['H', 'T'])
        
        if flip1 == 'H':
            count_first_heads += 1
        if flip2 == 'H':
            count_second_heads += 1
        if flip1 == 'H' and flip2 == 'H':
            count_both_heads += 1
    
    p_first = count_first_heads / n_trials
    p_second = count_second_heads / n_trials
    p_both = count_both_heads / n_trials
    p_product = p_first * p_second
    
    print(f"P(H₁) = {p_first:.4f}")
    print(f"P(H₂) = {p_second:.4f}")
    print(f"P(H₁ ∩ H₂) simulated = {p_both:.4f}")
    print(f"P(H₁) × P(H₂) = {p_product:.4f}")
    print(f"\nIndependent: {abs(p_both - p_product) < 0.01}")

test_independence_coin_flips()

Testing Independence

Example 2: Dependent Events

Drawing cards without replacement creates dependence:

First draw: P(Ace) = 4/52 Second draw given first was Ace: P(Ace) = 3/51 ≠ 4/52

The events are dependent because the first draw changes the composition of the deck.

python

def test_cards_dependence(n_trials=10000):
    """Show cards without replacement are dependent"""
    import random
    
    count_both_aces = 0
    count_first_ace = 0
    
    for _ in range(n_trials):
        deck = ['A']*4 + ['X']*48  # 4 aces, 48 others
        random.shuffle(deck)
        
        first = deck[0]
        second = deck[1]
        
        if first == 'A':
            count_first_ace += 1
            if second == 'A':
                count_both_aces += 1
    
    p_first_ace = count_first_ace / n_trials
    p_both_aces = count_both_aces / n_trials
    p_second_given_first = p_both_aces / p_first_ace if p_first_ace > 0 else 0
    
    print(f"P(first is ace) = {p_first_ace:.4f} (theoretical: {4/52:.4f})")
    print(f"P(second is ace | first is ace) = {p_second_given_first:.4f}")
    print(f"Theoretical P(2nd ace | 1st ace) = {3/51:.4f}")
    print(f"\nIf independent, would be {4/52:.4f}")
    print(f"But it's {p_second_given_first:.4f}, so DEPENDENT!")

test_cards_dependence()

Pairwise vs Mutual Independence

Pairwise Independence

Events $A_1, ..., A_n$ are pairwise independent if:

$P(A_i \cap A_j) = P(A_i) P(A_j) \quad \text{for all } i \neq j$

Mutual Independence

Events are mutually independent if for every subset $\{i_1, ..., i_k\}$ :

$P(A_{i_1} \cap \cdots \cap A_{i_k}) = P(A_{i_1}) \cdots P(A_{i_k})$

Key: Mutual independence is stronger than pairwise independence!

Common Mistake: Pairwise independent does NOT imply mutually independent!

Counterexample: Two fair coin flips, let:

$A$ = "first is H"
$B$ = "second is H"
$C$ = "exactly one H"

These are pairwise independent but NOT mutually independent: $P(A \cap B \cap C) = 0 \neq P(A)P(B)P(C) = 1/8$

Independence and Complements

Theorem: If $A$ and $B$ are independent, then:

$A$ and $B^c$ are independent
$A^c$ and $B$ are independent
$A^c$ and $B^c$ are independent

Proof (for $A$ and $B^c$ ):

$P(A \cap B^c) = P(A) - P(A \cap B) = P(A) - P(A)P(B) = P(A)(1 - P(B)) = P(A)P(B^c)$

Key Takeaways

Independence: $P(A \cap B) = P(A)P(B)$ or equivalently $P(A|B) = P(A)$
Test: Check if knowing one event changes probability of the other
Pairwise ≠ Mutual: Pairwise independence is weaker
Complements: Independence extends to complements
Dependence: Most real-world events are dependent!

Next Lesson: Advanced conditional probability problems including Monty Hall!

Concept	Definition	Example
Independence	$P(A \cap B) = P(A)P(B)$	Coin flips
Dependence	$P(A \cap B) \neq P(A)P(B)$	Cards without replacement
Pairwise	Each pair independent	$P(A_i \cap A_j) = P(A_i)P(A_j)$
Mutual	All subsets independent	Stronger than pairwise

IntroductionFocusStart Focus Mode

Independence DefinitionFocusStart Focus Mode