How a hypothesis test works: null, critical value, p-value

What is a hypothesis test?

A hypothesis is a statement about a population that we can check against data, such as “this strategy earns a positive excess return”. Hypothesis testing is the procedure that uses a sample to decide whether such a statement is plausible or whether the sample is too far from it to be chance. It answers one question: is the result real, or just sampling noise?

What are the six steps?

Memorize

Every test follows the same path, whatever statistic it uses.

1. State the hypotheses.
2. Identify the test statistic and its distribution.
3. Choose the significance level.
4. State the decision rule.
5. Collect the data and compute the statistic.
6. Decide, then make the economic or investment decision.

Hypothesis testfrom a claim to a decision

• 1. State the hypotheses Memorize H₀ (with the equals sign) and Hₐ
• 2. Identify the test statistic Memorize and its distribution
• 3. Choose the significance level Memorize α, usually 5%
• 4. State the decision rule Memorize critical value(s)
• 5. Compute the statistic Memorize from the sample
• 6. Decide Memorize reject or fail to reject, then the economic decision

What are the null and the alternative?

The null hypothesis (H₀) is the statement we try to reject. It is assumed true until the sample gives convincing evidence otherwise, and it always contains the equals sign (=, ≤ or ≥). The alternative hypothesis (Hₐ) is what we accept if we reject the null. Together they cover every possible value of the parameter.

The practical rule: what you hope to show goes in Hₐ. If an analyst suspects a fund beats 6%, the claim “above 6%” is the alternative, and the null is “at most 6%”.

Exam tip Write Hₐ first, straight from the claim. Then H₀ is whatever is left over, with the equals sign.

One-tailed or two-tailed?

The wording of Hₐ sets the number of tails. A two-tailed test asks whether the parameter is different from the hypothesized value in either direction, so the rejection region is split between both tails. A one-tailed test asks about one direction only and puts all of the rejection region in that tail.

The claim you want to showit goes in Hₐ; the equals sign stays in H₀

• Two-tailed test Memorize the claim is "different from"
  • H₀: μ = μ₀ versus Hₐ: μ ≠ μ₀ Memorize α split across both tails; z at 5%: ±1.96
• One-tailed test, right Memorize the claim is "greater than"
  • H₀: μ ≤ μ₀ versus Hₐ: μ > μ₀ Memorize all of α in the right tail; z at 5%: +1.645
• One-tailed test, left Memorize the claim is "less than"
  • H₀: μ ≥ μ₀ versus Hₐ: μ < μ₀ Memorize all of α in the left tail; z at 5%: −1.645

The one-tailed critical value is smaller in absolute terms than the two-tailed value at the same significance level, which makes a one-tailed test easier to pass in the direction you predicted. That is why the direction must come from the claim, not from the data.

How do you build the test statistic and the decision rule?

Memorize

The test statistic measures how far the sample result is from the hypothesized value, counted in standard errors.

$$\text{Test statistic} = \frac{\text{Sample statistic} - \text{Hypothesized value}}{\text{Standard error of the sample statistic}}$$

For a mean with an unknown population variance, the statistic follows a t-distribution with n − 1 degrees of freedom.

$$t = \frac{\bar{X} - \mu_0}{s \,/\, \sqrt{n}},\quad df = n - 1$$

The level of significance (α) is the probability of rejecting a true null that you are willing to accept, usually 5%. It fixes the critical values : the cut-off points that separate the rejection region from the rest. The confidence level is its complement.

$$\text{Confidence level} = 1 - \alpha$$

Reject H₀ if the statistic falls beyond a critical value. Otherwise fail to reject it.

Reject H₀ when

• Critical-value rule Memorize the test statistic is beyond the critical value
• p-value rule Memorize the p-value is less than α

Slide the statistic into the shaded tail and watch the decision flip. Then switch from one-tailed to two-tailed and see the critical value move out.

What does the p-value add?

The p-value is the smallest significance level at which the null can be rejected. For a right-tailed test it is the area in the tail beyond your statistic. Using it replaces the critical value: reject if the p-value is less than α.

In the example, t = 2.25 with 35 degrees of freedom has a one-tailed p-value of 0.015. So H₀ is rejected at 5% and at 2.5%, but not at 1%. A software package reports the p-value; the exam usually gives it or gives critical values, not both.

A result that clears the bar is statistically significant : it is unlikely to be chance alone at the chosen level. That says nothing yet about whether it matters in money terms.

Go deeper Statistical versus economic significance

An edge of 0.05% a month can be statistically significant in a long enough sample, yet vanish once trading costs, taxes and the risk of the strategy are counted. Always finish step 6 by asking whether the effect is large enough to act on after costs, and whether it would survive in a different period.

Go deeper Confidence intervals and two-tailed tests agree

A two-tailed test at 5% rejects H₀: μ = μ₀ exactly when μ₀ lies outside the 95% confidence interval for μ. The two tools make the same decision from different angles: the interval shows the range of plausible values, the test gives a yes or no.

Go deeper z or t?

Use a t-distribution when the population variance is unknown, which is almost always. The t critical values are larger for small samples, so the test is more cautious, and they approach the z values as n grows. With σ known, or in a large sample where the exam says so, use z.