How a hypothesis test works: four steps from claim to decision

What is a hypothesis test?

A hypothesis is a statement about a population that we can check against data, such as “this strategy earns a positive excess return”. Hypothesis testing is the procedure that uses a sample to decide whether such a statement is plausible or whether the sample is too far from it to be chance. It answers one question: is the result real, or just sampling noise?

What are the four steps?

Memorize

Every test follows the same path, whatever statistic it uses.

1. State the null and alternative hypotheses.
2. Identify the test statistic and its distribution, and compute it.
3. Specify the level of significance, α (the confidence level is 1 − α).
4. Establish the decision rule: reject $H_0$ if the test statistic is beyond the critical value; otherwise fail to reject it.

Hypothesis testfrom a claim to a decision

• 1. State the hypotheses Memorize H₀ with the equals sign; Hₐ is the claim
• 2. Identify the test statistic Memorize and its distribution; compute it
• 3. Specify the significance level Memorize α; confidence level 1 − α
• 4. Establish the decision rule Memorize reject H₀ if the statistic is beyond the critical value

What are the null and the alternative?

The null hypothesis ($H_0$) is the statement we try to reject. It is assumed true until the sample gives convincing evidence otherwise, and it always contains the equals sign (=, ≤ or ≥). The alternative hypothesis ($H_A$) is the claim to be proven, and the one the data support if we reject the null. The null is the status quo, no effect or no difference. The two are mutually exclusive and together cover every possibility.

The practical rule: what you hope to show goes in $H_A$. If an analyst suspects a fund beats 8%, the claim “above 8%” is the alternative, and the null is “at most 8%”.

Exam tip Write $H_A$ first, straight from the claim. Then $H_0$ is whatever is left over, with the equals sign.

What kinds of test are there?

Tests fall into three families, by what they compare.

What the test compares

• Against a known value Memorize a mean or variance vs a target, such as an 8% return
• Two unknown values Memorize two means or two variances, such as two funds
• Association Memorize a relationship, such as a correlation or independence

• Tests against a known value compare a mean or variance with a target: “does the fund’s mean return beat its 8% target?”
• Tests comparing two unknown values compare two means or two variances: “is fund A more volatile than fund B?”
• Tests of association ask whether a relationship exists: “do bond returns move with changes in the policy rate?”

One-tailed or two-tailed?

The wording of $H_A$ sets the number of tails. A two-tailed test asks whether the parameter is different from the hypothesized value in either direction, so the rejection region is split between both tails. A one-tailed test asks about one direction only and puts all of the rejection region in that tail: the right-tail test when the claim is “greater than”, the left-tail test when it is “less than”.

The claim you want to showit goes in Hₐ; the equals sign stays in H₀

• Two-tailed test Memorize the claim is "different from"
  • H₀: μ = μ₀ versus Hₐ: μ ≠ μ₀ Memorize α split across both tails; z at 5%: ±1.96
• One-tailed test, right Memorize the claim is "greater than"
  • H₀: μ ≤ μ₀ versus Hₐ: μ > μ₀ Memorize all of α in the right tail; z at 5%: +1.645
• One-tailed test, left Memorize the claim is "less than"
  • H₀: μ ≥ μ₀ versus Hₐ: μ < μ₀ Memorize all of α in the left tail; z at 5%: −1.645

Form	Example claim	Hypotheses
Right-tail	a fund’s mean return is above its 8% target	$H_0$: μ ≤ 8%; $H_A$: μ > 8%
Left-tail	a new trading algorithm lowers the average cost per trade	$H_0$: μ ≥ current cost; $H_A$: μ < current cost
Two-tailed	a share’s beta is different from 1	$H_0$: β = 1; $H_A$: β ≠ 1

The one-tailed critical value is smaller in absolute terms than the two-tailed value at the same significance level, which makes a one-tailed test easier to pass in the direction you predicted. That is why the direction must come from the claim, not from the data.

How do you build the test statistic and the decision rule?

Memorize

The test statistic measures how far the sample result is from the hypothesized value, counted in standard errors.

$$\text{Test statistic} = \frac{\text{Sample statistic} - \text{Hypothesized value}}{\text{Standard error of the sample statistic}}$$

For a mean with an unknown population variance, the statistic follows a t-distribution with n − 1 degrees of freedom (df).

$$t = \frac{\bar{X} - \mu_0}{s \,/\, \sqrt{n}},\quad df = n - 1$$

Every test in this module has its own statistic, and each has a lesson. This table is the map.

Test	Statistic	Degrees of freedom	Order matters?	Lesson
Single mean	t = (x̄ − μ₀) ÷ (s ÷ √n)	n − 1	no	t-tests
Single variance	χ² = (n − 1)s² ÷ σ₀²	n − 1	no	variance tests
Difference in means	t (pooled variance)	n₁ + n₂ − 2	no	t-tests
Difference in variances	F = s₁² ÷ s₂²	n₁ − 1 and n₂ − 1	no	variance tests
Paired differences	t = (d̄ − μ₀) ÷ (s_d ÷ √n)	n − 1	yes	t-tests
Correlation	t = r√(n − 2) ÷ √(1 − r²)	n − 2	yes	correlation test
Independence	χ² = ΣΣ(O − E)² ÷ E	(r − 1)(c − 1)	yes	independence test

The “order matters” column splits the tests in two. An order-independent test treats each sample as a set: shuffle the dates and the statistic does not change. An order-dependent test rests on how observations are paired, so re-pairing them changes the result.

The level of significance (α) is the probability of rejecting a true null that you are willing to accept, usually 5%. It fixes the critical values : the cut-off points that separate the rejection region from the rest. The confidence level is its complement.

$$\text{Confidence level} = 1 - \alpha$$

The decision rule is the final step. Reject $H_0$ if the statistic falls beyond a critical value. Otherwise fail to reject it.

“Big is bad for the null”: a test statistic far from zero means the data sit badly with $H_0$. In a two-tailed test, compare the absolute value of the statistic with the critical value; in a one-tailed test, the statistic must also lie in the tail named by $H_A$.

Reject H₀ when

• Critical-value rule Memorize the test statistic is beyond the critical value
• p-value rule Memorize the p-value is less than α

Exam tip Critical values come from tables or, more often in practice, from software such as a spreadsheet. The p-value lesson shows the functions.

Slide the statistic into the shaded tail and watch the decision flip. Then switch from one-tailed to two-tailed and see the critical value move out.

t or Z?

Use it

When the population standard deviation is unknown, which is almost always, the sample mean follows a t-distribution with n − 1 degrees of freedom; with σ known it is Z. The t-distribution approaches the normal as n grows, so the curriculum uses Z for large samples (n of 30 or more), a known σ or a normal population, and t for small samples (under 30) or an unknown σ. These sample-size rules do not apply to chi-square and F tests.

What does the p-value add?

The p-value is the smallest significance level at which $H_0$ would be rejected. In the example, t = 2.25 with 35 degrees of freedom has a one-tailed p-value of 0.015: reject at 5%, not at 1% (critical value 2.438). Reject whenever p < α. The p-value lesson covers its forms, its misuse (p-hacking) and how to get it from software.

A result that clears the bar is statistically significant : it is unlikely to be chance alone at the chosen level. That says nothing yet about whether it matters in money terms.

Go deeper Statistical versus economic significance

An edge of 0.05% a month can be statistically significant in a long enough sample, yet vanish once trading costs, taxes and the risk of the strategy are counted. After the four steps, ask whether the effect is large enough to act on after costs, and whether it would survive in a different period.

Go deeper Confidence intervals and two-tailed tests agree

A two-tailed test at 5% rejects $H_0$: μ = μ₀ exactly when μ₀ lies outside the 95% confidence interval for μ. The two tools make the same decision from different angles: the interval shows the range of plausible values, the test gives a yes or no.

Go deeper z or t? A quick example

Eighteen monthly returns of a new fund, σ unknown, returns roughly normal: use t with 17 degrees of freedom. The t critical values are larger than the z values for small samples, so the test is more cautious.