Sampling methods and sampling error

How should you pick a sample?

Examining every member of a group is often impossible or too costly, so analysts study a sample and draw conclusions about the whole group. A parameter is a value that describes the whole population, such as its mean μ or standard deviation σ. A sample statistic is the matching value computed from the sample, such as X̄ or s. The statistic is the estimate; the parameter is what you want to know.

The full set of rules for drawing the sample is the sampling plan . Its quality decides how far the statistic can be trusted.

What is sampling error?

Memorize

Sampling error is the gap between the statistic and the parameter it estimates. It arises because you looked at only part of the population, so it is the price paid for saving time and money.

$$\text{Sampling error} = \bar{X} - \mu$$

Which sampling methods are there?

The first split is whether chance controls the selection. Probability sampling gives every member a known chance of being picked, so the result can be representative and the error can be measured. Non-probability sampling depends on access or on the analyst’s judgment, so it carries a significant risk of an unrepresentative sample.

Sampling methodshow the sample is picked

• Probability sampling Memorize known chance of selection, error can be measured
  • Simple random Memorize every member equally likely
  • Systematic Use it every kth member
  • Stratified random Memorize sample within every group, in proportion
  • Cluster Memorize pick whole groups at random
    • One-stagesurvey every member of the chosen groups
    • Two-stagerandom subset inside each chosen group
• Non-probability sampling Memorize depends on access or judgment, bias risk
  • Convenience Memorize whoever is easy to reach
  • Judgmental Memorize hand-picked by the analyst

Method	How (an example)	Sampling error	Watch out
Simple random	draw 50 of 1,200 listed companies by random number	measurable	may miss small subgroups by chance
Systematic	every 24th company on an alphabetical list	measurable, close to random	hidden patterns in the list
Stratified random	split companies by region and size band, sample each cell in proportion	smallest for a given n	needs a good way to define strata
Cluster	pick 4 of 20 regional branches at random and survey every client there	larger than stratified	clusters may not resemble each other
Convenience	survey whoever answers this week’s newsletter	not measurable	bias
Judgmental	the analyst hand-picks “representative” firms	not measurable	the analyst’s own bias

How do the probability methods work?

Memorize

In simple random sampling every member of the population has the same chance of selection, as if names were drawn from a hat. It works best when the population is fairly uniform. When it is not, chance alone can leave out a small but important subgroup.

Systematic sampling takes every kth member of a list until the sample is big enough. It is practical when you cannot code every member in advance, and the result should be approximately random, unless the list hides a repeating pattern that lines up with k.

$$k = \frac{N}{n}$$

In stratified random sampling the population is divided into subpopulations (strata) by one or more criteria. A simple random sample is then drawn from each stratum, in proportion to its share of the population, and the pieces are pooled. Every stratum of interest is guaranteed a place in the sample, and the estimates are more precise (smaller variance) than from simple random sampling of the same size.

$$n_i = n \times \frac{N_i}{N}$$

In cluster sampling the population is divided into clusters, each ideally a miniature of the whole (often geographic). Some clusters are picked at random and used as a whole. If every member of a chosen cluster is surveyed, it is one-stage cluster sampling. If a random subset is drawn inside each chosen cluster, it is two-stage.

Population split into groups

• Stratified Memorize every group is sampled
  • Inside each group a random draw, in proportion to its size
  • Result every group represented, smallest error
• Cluster Memorize only some groups are sampled
  • Chosen groups taken whole, or sampled again inside
  • Result cheap and quick, larger error

The difference is easy to state. Stratified sampling samples within every group. Cluster sampling samples whole groups and skips the rest. For the same sample size, cluster sampling is usually less accurate, because the chosen clusters may not mirror the population. Its strength is cost: it is the cheapest and quickest probability plan for a very large population.

How are strata used to track an index?

Indexing means building a portfolio that mirrors an index. Owning every security is costly when the index has thousands, many of them illiquid. Holding a simple random sample would be cheap but might not match the index’s main risk drivers.

Stratified sampling fixes this. Divide the index into cells by the drivers that matter (for equities: region, size, sector; for bonds: duration, credit quality, sector, coupon), then hold a few securities in each cell, in proportion to the cell’s weight in the index.

What about the non-probability methods?

Convenience sampling picks whatever is easiest to reach. It is quick and cheap, which suits a pilot study, but the sample may not represent the population and its error cannot be measured.

Judgmental sampling means the analyst hand-picks members using knowledge and professional judgment. When time is short or expertise matters, such as a seasoned auditor choosing which transactions deserve scrutiny, it can reach the target group directly. The risk is the analyst’s own bias.

Can you pool data from different regimes?

Not safely. A larger sample helps only if every observation comes from the same distribution. If a fund changes strategy, or a policy shift changes how markets behave, data from before and after describe two different populations.

Go deeper Why stratified sampling has the smaller error

Part of the variation in a simple random sample comes from chance differences in how many members of each stratum happen to be drawn. Stratified sampling fixes the number from each stratum in advance, so that source of variation disappears and only the variation within strata remains. The gain is largest when the strata differ a lot from each other. When all strata look alike, stratifying adds little.

Go deeper More data, diminishing returns

Sampling error falls quickly as the sample grows, then flattens. Going from 10 to 100 observations transforms the estimate. Going from 1,000 to 1,100 barely changes it, while the cost keeps rising. Time-series samples add a second risk: a longer history raises the chance of mixing regimes, so more years is not always better.

Exam tip Stratified: every group, some members. Cluster: some groups, all members (or a subset).