Bootstrap and jackknife: resampling to estimate a standard error

How do you measure precision without a formula?

For a sample mean, the standard error is s ÷ √n. For a median, a percentile or a ratio such as the Sharpe ratio, no such simple formula exists. Resampling gets around this. It uses the one sample you hold as a stand-in for the population, builds many new samples from it, computes the statistic in each, and treats the spread of those results as the statistic’s sampling variability.

The pattern of results is an empirical sampling distribution : built from computation on observed data, not derived from a formula. Two methods are examined: the bootstrap and the jackknife.

Resamplingnew samples built from the one you have

• Bootstrap Memorize random, with replacement
  • Resample size Memorize n, the same as the original
  • Number of resamples B your choice, often 1,000 or more
  • Result varies a little from run to run
• Jackknife Memorize leave one out, no replacement
  • Resample size n − 1
  • Number of resamples Memorize exactly n
  • Result is fixed Memorize same answer every run

How does the bootstrap work?

Memorize

The bootstrap draws repeated random samples from the original data, each the same size as the original, using sampling with replacement : each item drawn is put back, so it can be drawn again. In any one resample some observations appear several times and others not at all.

Bootstrapfrom one sample of n observations

• 1. Resample Memorize draw n observations with replacement
• 2. Compute the statistic (mean, median, ratio) for that resample
• 3. Repeat B times for example 1,000
• 4. Take the spread Memorize the standard deviation of the B results is the standard error

For a sample mean, the bootstrap standard error is the standard deviation of the B resample means, using a B − 1 divisor.

$$s_{\bar{X}} = \sqrt{\frac{1}{B-1}\sum_{b=1}^{B}(\hat\theta_b - \bar\theta)^2}$$

Here θ̂_b is the statistic from resample b (a resample mean) and θ̄ is the average of all B of them. The method needs no distribution assumption and works for any statistic. It is used in finance for historical simulation and for judging a strategy’s performance against a benchmark. Because the draws are random, two runs give slightly different answers.

How does the jackknife work?

Memorize

The jackknife leaves out one observation at a time, without putting it back. Drop observation 1 and compute the statistic on the other n − 1; return it, drop observation 2, and so on. A sample of n gives exactly n results. Nothing is random, so every run gives the same answer. It is used for the standard error, bias and confidence interval of an estimator, and suits small samples because it needs few computations.

To turn the n results into a standard error, a scaling factor is needed, because leave-one-out results sit much closer together than independent resamples would.

$$SE = \sqrt{\frac{n-1}{n}\sum_{i=1}^{n}(\theta_i - \bar\theta)^2}$$

Which method should you use?

	Bootstrap	Jackknife
Sample drawn	n, with replacement	n − 1, leaving one out, no replacement
Number of resamples	chosen (often 1,000+)	n
Random?	yes, results vary slightly	no, same result every time
Typical use	standard error, confidence intervals, complex statistics	standard error and bias, small samples

Exam tip Bootstrap: with replacement, random, B of your choosing. Jackknife: leave one out, n of them, nothing random.

Neither method creates information. Both reuse the observations in your sample, so a sample that is unrepresentative gives an unrepresentative answer, however many resamples you run.

Go deeper Bootstrap for simulation versus bootstrap for inference

The simulation module used the bootstrap to generate possible future paths of returns. Here the mechanics are identical (draw from the observed data with replacement) but the goal differs: instead of building scenarios, you compute a statistic on each resample and study how it varies, to learn how precise it is.

Go deeper Why the sample can stand in for the population

The true sampling distribution would need many fresh samples from the real population, which you do not have. The bootstrap replaces the unknown population with the empirical distribution of your sample, and draws from it repeatedly. If the sample is representative, the variation across resamples approximates the variation across real samples.