The central limit theorem and the standard error of the mean

Why does the sample mean have a distribution?

The law of large numbers says the sample mean settles on μ as n grows; the central limit theorem adds the shape of its distribution on the way.

Draw one sample and you get one sample mean. Draw another and you get a different one. The sample mean is therefore a random variable, and the pattern of values it takes across all possible samples of the same size is its sampling distribution . To judge how close X̄ is likely to be to μ, you need to know the shape, center and spread of that distribution. The central limit theorem supplies all three.

What does the central limit theorem say?

Memorize

The central limit theorem (CLT) says that, for any population with mean μ and finite variance σ², the sampling distribution of the sample mean from random samples of size n is approximately normal with mean μ (the population mean ) and variance σ² ÷ n, once n is large.

The theorem needs only a finite mean and a finite variance, and it works for discrete and continuous populations alike. In standardized form, Z = (x̄ − μ) ÷ (σ ÷ √n) approaches the standard normal distribution, N(0, 1), as n grows.

Sample mean, X̄large n, any population with finite variance

• Shape Memorize approximately normal as n grows; more n needed the more skewed the population
• Center Memorize equals the population mean μ
• Spread Memorize variance σ² ÷ n
  • Standard error Memorize σ ÷ √n, or s ÷ √n when σ is unknown

$$\bar{X} \sim N\!\left(\mu,\ \frac{\sigma^2}{n}\right)$$

Press the draw button repeatedly with a small n, then a large one. The population stays skewed, but the sample means pile up in a bell shape that narrows as n grows.

What does the theorem not say?

It does not say the population, or the individual observations, become normal. A skewed return series stays skewed however many observations you collect. What becomes normal is the distribution of the average.

It also does not say the sample mean moves toward a different center. The center is μ for every n; larger n only tightens the spread. And “large” depends on the population: for a symmetric, single-peaked population 4 or 5 observations can be enough, a skewed one needs at least 25 to 30, and an extremely skewed one more. A sample above 30 is the common working guide.

Why it matters: the normal distribution is well tabulated. Because of the CLT we can make probability statements about a population mean from a sample mean, even when returns are skewed or their distribution is unknown. That is the basis of the confidence intervals and hypothesis tests later in this module.

How precise is the sample mean?

Memorize

The variance of X̄ is σ² ÷ n. Its square root, the standard deviation of the sample mean, is the standard error of the sample mean .

$$\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$$

In practice σ is rarely known, so we replace it with the sample standard deviation s, computed with an n − 1 divisor.

$$s_{\bar{X}} = \frac{s}{\sqrt{n}}$$

Standard error of the sample mean

• Population σ known rare
  • σ ÷ √n Memorize
• Population σ unknown the usual case
  • s ÷ √n Memorize s is the sample standard deviation (n − 1 divisor)

Exam tip The standard error divides by √n. If you divided by n, or used the variance, you have fallen for an option.

Standard error or standard deviation?

They are different concepts and are not interchangeable.

	Standard deviation	Standard error
Measures	spread of the observations around their mean	spread of the sample mean around the population mean
Answers	how variable are the data?	how precise is my estimate?
Effect of larger n	settles near the true σ	shrinks toward zero
Use	describing	inferring

What makes a good estimator?

Use it

The sample mean x̄ is a point estimate of μ: one number standing in for the unknown population value. Wrap a range around it and you have an interval estimate , the subject of the next lesson. The sample mean is an unbiased and consistent estimator of μ, and its precision is the standard error. The central limit theorem is used in two ways: estimation (a point estimate such as x̄, and an interval estimate around it) and hypothesis testing.

Statisticians judge an estimator, the rule that turns a sample into an estimate, by a short list of properties.

A good estimatora rule for turning a sample into an estimate of a parameter

• Unbiased Use it its expected value equals the parameter: E[x̄] = μ
• Minimum-variance unbiased estimator (MVUE) Know the idea unbiased, with the smallest standard deviation of all unbiased estimators
• Best linear unbiased estimator (BLUE) Know the idea unbiased and the most efficient among linear combinations of the data
• Consistent Use it converges to the true value as n grows; x̄ is consistent
• Robust Use it stays reliable with outliers or non-normal data, for example by trimming

An unbiased estimator is right on average: across all possible samples its expected value equals the parameter. Among unbiased estimators, the minimum-variance unbiased estimator (MVUE) has the smallest standard deviation of all, so its estimates scatter least around the truth. The best linear unbiased estimator (BLUE) wins the same contest only among estimators that are linear combinations of the data, a narrower field. A consistent estimator converges to the true value as the sample grows, and a robust estimator stays reliable when the data contain outliers or are not normal, for example by trimming extreme values before averaging.

Exam tip MVUE beats BLUE: “best” among all unbiased estimators is the stronger claim than “best” among linear ones.

How big must the sample be?

Rearrange the formula: to reach a target standard error, take n = (s ÷ standard error)². Precision is expensive. Halving the target means four times as many observations.

$$n = \left(\frac{s}{s_{\bar{X}}}\right)^2$$

Exam tip The standard error is the building block of the next lesson: a confidence interval is x̄ plus or minus a critical value times s ÷ √n.

Go deeper Why there is no single magic sample size

How fast the sample mean becomes normal depends on how far the population is from normal. Symmetric, single-peaked data converge with a handful of observations. Skewed returns need at least 25 to 30, and heavily skewed or fat-tailed returns may need hundreds. Check the shape of the data rather than treating any one number as a guarantee.

Go deeper A common misuse of the theorem

An analyst holds 40 monthly returns from a strategy that sells options: small gains most months, a rare large loss. The analyst argues that, because of the CLT, next month’s return is normally distributed. It is not. The theorem describes the average of the 40 returns, not any single month, and one month’s return keeps the population’s skew. Even for the average, a population this lopsided may need far more than 40 observations before the normal approximation is reliable.