Probability: compound events and conditional probability

Probability as a fraction of outcomes

When every outcome is equally likely,

$$P(\text{event}) = \frac{\text{number of favourable outcomes}}{\text{total number of outcomes}}.$$

A probability is always between 0 (impossible) and 1 (certain). Drawing one marble from a bag of 5 red, 3 blue and 2 green, $P(\text{blue}) = \frac{3}{10}$. Everything in this lesson either counts outcomes like this or combines probabilities you already have.

"Not": the complement

The probability that an event does not happen is

$$P(\text{not } A) = 1 - P(A).$$

This is the tool for "at least one". Counting all the ways to get at least one success is messy; the only way to fail is to get no successes, which is usually one simple case. So $P(\text{at least one}) = 1 - P(\text{none})$.

"Or": the addition rule

$$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$$

The subtraction removes outcomes counted in both, exactly as in a Venn diagram. If $A$ and $B$ cannot happen together (they are mutually exclusive), $P(A \text{ and } B) = 0$ and the probabilities simply add. Rolling a fair six-sided die, "even" and "greater than 4" overlap at 6, so $P = \frac{3}{6} + \frac{2}{6} - \frac{1}{6} = \frac{4}{6} = \frac{2}{3}$; listing the outcomes 2, 4, 5, 6 confirms it.

"And": the multiplication rule

Two events are independent if knowing that one happened does not change the probability of the other: separate coin tosses, separate dice, draws with replacement. For independent events,

$$P(A \text{ and } B) = P(A) \times P(B).$$

When the first event changes the situation for the second, the events are dependent, and you multiply by the probability of the second given the first:

$$P(A \text{ and } B) = P(A) \times P(B \mid A).$$

Drawing without replacement is the usual case. From the bag of 10 marbles above, the probability that two draws are both red is $\frac{5}{10} \times \frac{4}{9}$, because after one red marble is gone, 4 of the 9 left are red.

Do not confuse "independent" with "mutually exclusive". Mutually exclusive events with positive probabilities are never independent: if one happens, the other's probability drops to 0.

Probability by counting

When the process is a selection, counting methods give the probability directly. The chance that a random group of 2 from the 10 marbles is both red is

$$\frac{\binom{5}{2}}{\binom{10}{2}} = \frac{10}{45} = \frac{2}{9},$$

the same as $\frac{5}{10} \times \frac{4}{9} = \frac{20}{90} = \frac{2}{9}$. Having two routes gives you a check.

When an outcome can happen in more than one order (one red and one blue, say), multiplying along one order counts only that order. Either add the orders, or count unordered groups.

Conditional probability

$P(A \mid B)$, read "the probability of $A$ given $B$", is the probability of $A$ when you already know $B$ happened. Knowing $B$ shrinks the set of possible outcomes to $B$ itself:

$$P(A \mid B) = \frac{P(A \text{ and } B)}{P(B)}.$$

With a two-way table of counts, this is just a fraction of one row or one column: the count in the cell for "$A$ and $B$" divided by the total for $B$. The word after "given" (or after "of those who", "among") names the total you divide by.

$P(A \mid B)$ and $P(B \mid A)$ are different numbers in general, because they divide by different totals. The probability that a randomly chosen basketball player is tall is high; the probability that a randomly chosen tall person plays basketball is low.

Two events are independent exactly when $P(A \mid B) = P(A)$: knowing $B$ tells you nothing new about $A$. A two-way table lets you test this by comparing a row's proportion with the overall proportion.

A routine for any question

1. Decide what one outcome is, and whether outcomes are equally likely.
2. Translate the words: "and" multiplies, "or" adds (minus the overlap), "not" and "at least one" use the complement, "given" divides by a smaller total.
3. Check independence before multiplying.
4. Sanity-check: the answer lies between 0 and 1, and an "and" probability is no larger than either event alone.