Sampling methods and sampling error

How should you pick a sample?

Examining every member of a group is often impossible or too costly, so analysts study a sample and draw conclusions about the whole group. A parameter is a value that describes the whole population, such as its mean μ or standard deviation σ. A sample statistic is the matching value computed from the sample, such as X̄ or s. The statistic is the estimate; the parameter is what you want to know.

The full set of rules for drawing the sample is the sampling plan . Its quality decides how far the statistic can be trusted.

What is sampling error?

Memorize

Sampling error is the gap between the statistic and the parameter it estimates. It arises because you looked at only part of the population, so it is the price paid for saving time and money. The sampling distribution of a statistic is the distribution of all the values it can take across samples of the same size from the same population.

$$\text{Sampling error} = \bar{X} - \mu$$

Which sampling methods are there?

The first split is whether chance controls the selection. Probability sampling gives every member an equal chance of being picked, so the result can be representative and the error can be measured. Non-probability sampling depends on access or on the analyst’s judgment, so it carries a significant risk of an unrepresentative sample.

Sampling methodshow the sample is picked

• Probability sampling Memorize equal chance of selection, so the error can be measured
  • Simple random Memorize every member equally likely
  • Systematic Use it every kth member
  • Stratified random Memorize sample within every group, in proportion
  • Cluster Memorize pick whole groups at random
    • One-stagesurvey every member of the chosen groups
    • Two-stagerandom subset inside each chosen group
• Non-probability sampling Memorize depends on access or judgment, bias risk
  • Convenience Memorize whoever is easy to reach
  • Judgmental Memorize hand-picked by the analyst

Method	How (an example)	Sampling error	Watch out
Simple random	draw 50 of 1,500 listed companies by random number	measurable	may miss small subgroups by chance
Systematic	every 30th company on an alphabetical list	measurable, close to random	hidden patterns in the list
Stratified random	split companies by region and size band, sample each cell in proportion	smallest for a given n	needs a good way to define strata
Cluster	pick 4 of 20 regional branches at random and survey every client there	larger than stratified	clusters may not resemble each other
Convenience	survey whoever answers this week’s newsletter	not measurable	bias
Judgmental	the analyst hand-picks “representative” firms	not measurable	the analyst’s own bias

How do the probability methods work?

Memorize

In simple random sampling every member of the population has the same chance of selection, as if names were drawn from a hat. It works best when the population is fairly uniform. When it is not, chance alone can leave out a small but important subgroup.

Systematic sampling takes every kth member of a list until the sample is big enough. It is practical when you cannot code every member in advance, and the result should be approximately random, unless the list hides a repeating pattern that lines up with k.

$$k = \frac{N}{n}$$

In stratified random sampling the population is divided into subpopulations (strata) by one or more criteria. A simple random sample is then drawn from each stratum, in proportion to its share of the population, and the pieces are pooled. Every stratum of interest is guaranteed a place in the sample, and the estimates are more precise (smaller variance) than from simple random sampling of the same size.

$$n_i = n \times \frac{N_i}{N}$$

In cluster sampling the population is divided into clusters, each ideally a miniature of the whole (often geographic). Some clusters are picked at random and used as a whole. If every member of a chosen cluster is surveyed, it is one-stage cluster sampling. If a random subset is drawn inside each chosen cluster, it is two-stage.

Population split into groups

• Stratified Memorize every group is sampled
  • Inside each group a random draw, in proportion to its size
  • Result every group represented, smallest error
• Cluster Memorize only some groups are sampled
  • Chosen groups taken whole, or sampled again inside
  • Result cheap and quick, larger error

The difference is easy to state. Stratified sampling samples within every group. Cluster sampling samples whole groups and skips the rest. For the same sample size, cluster sampling is usually less accurate, because the chosen clusters may not mirror the population. Its strength is cost: it is the cheapest and quickest probability plan for a very large population.

What about the non-probability methods?

Convenience sampling picks whatever is easiest to reach. It is quick and cheap, which suits a pilot study, but the sample may not represent the population and its error cannot be measured.

Judgmental sampling means the analyst hand-picks members using knowledge and professional judgment. When time is short or expertise matters, such as a seasoned auditor choosing which transactions deserve scrutiny, it can reach the target group directly. The risk is the analyst’s own bias.

When does the central limit theorem apply to a sample?

Use it

The central limit theorem rests on a representative, randomly drawn sample, so the sampling method decides whether it can be used.

Method	The central limit theorem…
Simple random, systematic	applies as n grows; above about 30 is the usual guide
Stratified random	applies within each stratum and to the combined mean as the strata samples grow; slower if strata differ a lot
Cluster	applies when each cluster is large, not when there are many small clusters; may need a bigger sample because members of a cluster tend to be alike (intra-cluster correlation )
Convenience, judgmental	does not reliably apply: the sample is not representative

Go deeper Why stratified sampling has the smaller error

Part of the variation in a simple random sample comes from chance differences in how many members of each stratum happen to be drawn. Stratified sampling fixes the number from each stratum in advance, so that source of variation disappears and only the variation within strata remains. The gain is largest when the strata differ a lot from each other. When all strata look alike, stratifying adds little.

Go deeper How fast does sampling error fall?

Experiments on daily return data show the mean absolute error (MAE) of the sample mean falling quickly as the sample grows, for random and stratified samples alike. Stratified samples start with the smaller error, but the advantage shrinks as n grows, and beyond about 200 observations extra data add little accuracy.

Exam tip Stratified: every group, some members. Cluster: some groups, all members (or a subset).