Standard deviation, quartiles and boxplots

Spread: how far values sit from the centre

Two lists can share a mean and still look nothing alike: 28, 29, 30, 31, 32 and 10, 20, 30, 40, 50 both have mean 30, but the second is far more spread out. The range (greatest minus least) catches this, yet it depends on only two values. Standard deviation uses every value: it is a typical distance of the data from the mean.

Computing a standard deviation

For a list of $n$ values with mean $\bar{x}$:

1. Find the mean.
2. Subtract the mean from each value to get its deviation.
3. Square each deviation (so negatives do not cancel positives).
4. Average the squares. This average is the variance.
5. Take the square root. That is the standard deviation.

$$\sigma = \sqrt{\frac{(x_1 - \bar{x})^2 + (x_2 - \bar{x})^2 + \cdots + (x_n - \bar{x})^2}{n}}$$

Some textbooks divide by $n - 1$ instead of $n$ when the data are a sample. This course divides by $n$ throughout; when two lists of the same length are compared, the choice never changes which spread is larger.

For 3, 5, 7, 9, 11: the mean is 7; the deviations are $-4, -2, 0, 2, 4$; the squares are 16, 4, 0, 4, 16, which add to 40; the variance is $\frac{40}{5} = 8$; and $\sigma = \sqrt{8} = 2\sqrt{2} \approx 2.83$.

Reasoning about standard deviation without computing

Most comparison questions can be settled with these facts.

• The standard deviation is never negative. It is 0 exactly when every value is the same.
• The further the values sit from their mean, the larger it is. Pulling values towards the mean shrinks it; pushing them away grows it.
• Adding the same number to every value does not change it: the whole list slides, but the distances to the mean stay the same. 13, 15, 17, 19, 21 has the same standard deviation as 3, 5, 7, 9, 11.
• Multiplying every value by a positive number $k$ multiplies the standard deviation by $k$. Tripling 3, 5, 7, 9, 11 gives a standard deviation of $3 \times 2\sqrt{2} = 6\sqrt{2} \approx 8.49$.
• Adding a new value equal to the mean lowers the standard deviation (unless it is already 0), because a value with zero deviation joins the average of the squares.

Quartiles and the interquartile range

Quartiles split the ordered data into four parts with roughly equal numbers of values. Conventions differ slightly between textbooks, and for small lists they can give slightly different quartiles. This course always uses the halves method:

• $Q_2$ is the median of the whole list.
• Split the ordered list into a lower half and an upper half. If the number of values is odd, leave the median out of both halves.
• $Q_1$ is the median of the lower half; $Q_3$ is the median of the upper half.

For 2, 5, 7, 10, 12, 13, 18, 25 (eight values), the median is $\frac{10 + 12}{2} = 11$. The lower half 2, 5, 7, 10 gives $Q_1 = \frac{5 + 7}{2} = 6$; the upper half 12, 13, 18, 25 gives $Q_3 = \frac{13 + 18}{2} = 15.5$.

The interquartile range is $\text{IQR} = Q_3 - Q_1$, here $15.5 - 6 = 9.5$. It measures the spread of the middle half of the data, so, like the median, it hardly reacts to outliers. Shifting every value leaves the IQR unchanged; multiplying every value by a positive $k$ multiplies it by $k$.

Percentiles

The $p$th percentile is a value at or below which about $p\%$ of the data lie. $Q_1$, the median and $Q_3$ are roughly the 25th, 50th and 75th percentiles. A percentile is a position in the data, not a percent score: being at the 90th percentile of a test means scoring at or above about 90% of test takers, whatever the raw mark.

Boxplots and the five-number summary

A boxplot draws five numbers: the minimum, $Q_1$, the median, $Q_3$ and the maximum. The box runs from $Q_1$ to $Q_3$ with a line at the median, and a whisker runs from each end of the box to the extreme value. (Some boxplots stop a whisker short and mark unusually distant values as separate points.) Read it as a table:

Minimum	$Q_1$	Median	$Q_3$	Maximum
12	17	22	30	41

Each of the four sections, whisker, half-box, half-box, whisker, holds about a quarter of the values, however long or short it looks. A long section means the values there are spread out, not that there are more of them. A boxplot shows the range and the IQR directly, but it does not show the mean, the standard deviation or the individual values.