Basic Probability
Learn sample spaces, events, probability axioms, the complement rule, addition rule, and multiplication rule.
Why Probability?
Probability is the engine that powers all of statistics. Without it, we can describe data (descriptive statistics), but we can't make claims beyond our data (inferential statistics).
At its core, probability answers one question: How likely is something to happen? But formalizing this simple question took centuries of brilliant minds â and the framework they built is what lets us do everything from testing drugs to predicting weather.
Sample Space and Events
Any process whose outcome is uncertain. Rolling a die, flipping a coin, measuring tomorrow's temperature, or checking if a patient recovers.
The set of all possible outcomes of an experiment.
- Coin flip: S = {Heads, Tails}
- Single die roll: S = {1, 2, 3, 4, 5, 6}
- Two coin flips: S = {HH, HT, TH, TT}
- Temperature tomorrow: S = all real numbers (continuous)
A subset of the sample space â a collection of outcomes we're interested in.
Examples with a die roll:
- Event A = "rolling an even number" = {2, 4, 6}
- Event B = "rolling greater than 4" = {5, 6}
- Event C = "rolling a 7" = {} (impossible event â empty set)
- Event D = "rolling any number" = {1, 2, 3, 4, 5, 6} (certain event)
Probability: The Rules
Probability assigns a number between 0 and 1 to every event, following three axioms established by Kolmogorov in 1933:
From these three axioms, everything else follows:
- P(A) = 0 means A is impossible
- P(A) = 1 means A is certain
- P(A) = 0.5 means A is equally likely to happen or not
For equally likely outcomes (fair dice, fair coins):
P(rolling a 3) = 1/6 â 0.167
P(rolling even) = P({2,4,6}) = 3/6 = 1/2
P(rolling > 4) = P({5,6}) = 2/6 = 1/3
The Complement Rule
The complement of event A (written A' or Aá¶) is "everything that's NOT A."
This is incredibly useful when it's easier to calculate the probability of something NOT happening:
Problem: What's the probability of rolling at least one 6 in four dice rolls?
Direct approach: Count all ways to get at least one 6... complicated.
Complement approach:
- P(no 6 on one roll) = 5/6
- P(no 6 on four rolls) = (5/6)⎠= 625/1296 â 0.482
- P(at least one 6) = 1 - 0.482 = 0.518
The complement turned a hard problem into an easy one!
"At least one" problems are almost always easier to solve using the complement: P(at least one) = 1 - P(none).
Addition Rule: P(A or B)
What's the probability that event A or event B happens (or both)?
We subtract P(A â© B) because outcomes in both A and B would be counted twice otherwise.
Special case â mutually exclusive events (A and B can't both happen):
Draw one card from a standard 52-card deck.
- A = drawing a King (4 kings â P(A) = 4/52)
- B = drawing a Heart (13 hearts â P(B) = 13/52)
- A â© B = King of Hearts (1 card â P(Aâ©B) = 1/52)
P(King or Heart) = 4/52 + 13/52 - 1/52 = 16/52 â 0.308
Without subtracting, we'd count the King of Hearts twice.
Multiplication Rule: P(A and B)
What's the probability that both A and B happen?
For independent events (one doesn't affect the other):
For dependent events (we'll explore this more in the conditional probability lesson):
Flip a coin and roll a die simultaneously.
- P(Heads) = 1/2
- P(rolling 6) = 1/6
P(Heads AND 6) = 1/2 Ă 1/6 = 1/12 â 0.083
The coin has no effect on the die, so we just multiply.
Common Probability Mistakes
1. The Gambler's Fallacy "I've flipped 5 heads in a row, so tails is due!" No. Each flip is independent. The coin has no memory. P(tails on the 6th flip) = 1/2, always.
2. Confusing "or" and "and" "Or" makes events more likely (wider net). "And" makes events less likely (stricter requirement). P(A or B) â„ P(A) â„ P(A and B).
3. Forgetting to subtract overlap When events can overlap, P(A or B) â P(A) + P(B). You must subtract the intersection.
4. Treating dependent events as independent Drawing two cards WITHOUT replacement: the first draw changes the deck. P(2nd King | 1st was King) = 3/51, not 4/52.
The Birthday Problem: In a room of just 23 people, there's a >50% chance two share a birthday. Our intuition about probability is notoriously terrible â this is why we need formal tools.