Parametric vs non-parametric tests
Parametric tests assume a distribution, usually normal, and test parameters such as a mean or a variance. Non-parametric tests assume no particular distribution, so they suit ranked, categorical or fat-tailed data, outliers and small samples. They are more robust but less powerful when the parametric assumptions hold.
Watch the lesson 10 min
The video follows this page: the same examples, then the checkpoint questions.
Before this: How a hypothesis test works: four steps from claim to decision
On the exam
- Compare and contrast parametric and non-parametric tests, describe situations in which each is the more appropriate type of test, construct appropriate hypothesis tests, and interpret the results
Compare and contrast parametric and non-parametric tests and describe when each is more appropriate. Expect a data description (ranks, categories, fat tails, a known normal distribution) and the choice it implies, and the trade-off: robustness against power.
How deeply to learn each part
- Know the idea
- Understand it in general. It frames other topics, but you will not calculate with it.
- Use it
- Understand it well enough to solve a question with it. Recognize when it applies.
- Memorize
- Learn it by heart. The exam will not give it to you.
You know this one. The full lesson is folded away. The takeaways, formulas and checkpoint below are your quick refresher, and the checkpoint is a fair test of whether you still do.
You tested strong here. The full lesson is folded away. The takeaways, formulas and checkpoint below are your quick refresher, and the checkpoint is a fair test of whether you still know it.
What makes a test parametric?
A parametric test assumes the data follow a particular distribution, often the normal, and meet conditions on randomness, sample size and variance; it uses parameters such as the mean and variance. A non-parametric test makes no assumption about the distribution. Every test in this module up to the t-test of correlation is parametric.
Many non-parametric tests work on ranks (the order of the observations) rather than on the raw numbers. Ranks suit ordinal data , which carry order but not size, such as credit grades. Counts suit categorical data , which are labels such as sector or rating band.
When should you switch?
Memorize
There are five situations.
- Distribution unknown or assumptions fail Memorize fat tails, the central limit theorem does not rescue a small or skewed sample
- Outliers Memorize extremes distort the mean and variance
- Ranked or ordinal data Memorize fund rankings, credit grades, scores
- Categorical data Memorize sector, rating band, style
- Small samples Memorize too few observations to trust a normal approximation
- Distribution unknown or assumptions fail. Fat tails, or a small or skewed sample that the central limit theorem cannot rescue, leave no valid t-test.
- Outliers. One extreme month can drag a mean and a variance, so a test based on ranks is steadier.
- Ranked or ordinal data. Fund rankings or credit grades carry order but not size, so means are meaningless.
- Categorical data. Companies counted by sector and rating band have no mean to test.
- Small samples. With too few observations, a normal approximation cannot be trusted.
When the distribution is known, or the central limit theorem’s assumptions hold, use the parametric test.
Sort each scenario into its bucket on the first tab. On the second, pick the test that fits the data. Notice how often the trigger is one word: “ranks”, “outlier”, “categories”.
What does robustness cost?
Non-parametric tests have robustness : they keep working when the data misbehave. The price is power. When the parametric assumptions do hold, the parametric test has more power (the power of a test is its chance of detecting a real effect), so switching without a reason throws information away. Reports often show both, so you can see how sensitive the conclusion is to the assumptions.
Go deeper Why ranking loses information
Replace 0.1%, 0.2% and 9.0% with ranks 1, 2 and 3 and the huge size of the last observation vanishes. That protects you from an outlier, and it also hides genuine large effects. So a non-parametric test is a safeguard, not an upgrade.
Which non-parametric tests does the exam use?
Memorize
Two: the Spearman rank correlation test, the partner of the t-test of a Pearson correlation, and the chi-square test of independence, for two categorical variables in a contingency table, which has no parametric partner.
- Spearman rank correlation test Memorize ranks; monotonic association
- Chi-square test of independence Memorize counts in a contingency table
| Data | Parametric test | Non-parametric test |
|---|---|---|
| Relationship between two numeric variables | t-test of Pearson correlation | Spearman rank test |
| Two categorical variables | none | chi-square test of independence |
Exam tip Same name, different family: the chi-square test of a single variance is parametric (it assumes a normal population); the chi-square test of independence is non-parametric.
Memorize this
MemorizeWhen to go non-parametric
| Trigger | Example |
|---|---|
| Fat tails or unknown distribution | skewed, crisis-prone returns |
| Outliers | one extreme month |
| Ranks or ordinal data | fund rankings, credit grades |
| Categorical data | sector by rating band |
| Small sample | eight quarters of data |
Memory hook FORCS: Fat tails, Outliers, Ranks, Categories, Small samples.
Know it cold Non-parametric = more robust, less powerful when the parametric assumptions hold.
MemorizeThe exam's non-parametric tests
| Data | Parametric test | Non-parametric test |
|---|---|---|
| Two numeric variables | t-test of the Pearson correlation | Spearman rank correlation test |
| Two categorical variables | none | chi-square test of independence |
Memory hook Same name, different family: the chi-square test of a single variance is parametric; the chi-square test of independence is not.
Know it cold When the distribution is known or the central limit theorem holds, use the parametric test.
- Distribution unknown or assumptions fail Memorize fat tails, the central limit theorem does not rescue a small or skewed sample
- Outliers Memorize extremes distort the mean and variance
- Ranked or ordinal data Memorize fund rankings, credit grades, scores
- Categorical data Memorize sector, rating band, style
- Small samples Memorize too few observations to trust a normal approximation
- Spearman rank correlation test Memorize ranks; monotonic association
- Chi-square test of independence Memorize counts in a contingency table
How it is asked
-
Ranks only
The analyst has only rankings (1 to 15) of managers from two consultants. Test of association?
Non-parametric, rank-based. Trap: a parametric t-test of the correlation.
-
Cost of switching
The parametric assumptions fully hold, but a non-parametric test is used. Effect?
Lower power. Trap: "more power because fewer assumptions".
-
Which test is non-parametric?
Chi-square test of a variance or chi-square test of independence?
Independence. The variance test assumes normality, so it is parametric.
-
Pick by the data
Returns with fat tails and two extreme months.
Non-parametric. A distribution known to be normal is the trap.
Exam traps
- Thinking non-parametric tests are always better because they assume less. When the assumptions hold, they have less power.
- Using a parametric test on rank data. Ranks do not support means and variances.
- Calling the chi-square test of independence parametric because the chi-square test of a variance is. Only the variance test assumes a normal population.
Checkpoint
4 exam-style questions. About 90 seconds each.
1An analyst has only the rankings (1 to 15) of fund managers by two consultants. A test of their association should most appropriately be:
2When the assumptions of a parametric test are fully met, using a non-parametric test instead will most likely:
3Which situation most favors a non-parametric test over a parametric one?
4Which of the following is a non-parametric test?
Key takeaways
- Parametric: about parameters, with a distribution assumed.
- Non-parametric: no distribution assumed; suits ranks, categories, fat tails, outliers and small samples.
- The exam's non-parametric tests: Spearman rank correlation and the chi-square test of independence.
- If the parametric assumptions hold, the parametric test is more powerful.
All Quantitative Methods formulas, every lesson
Practice exam-style questions on Estimation and Hypothesis Testing
Questions about this lesson?
Where this comes back
- Spearman rank correlation: the non-parametric test Estimation and Hypothesis Testing
- Chi-square test of independence with a contingency table Estimation and Hypothesis Testing
Questions candidates ask
What is the difference between parametric and non-parametric tests?
A parametric test assumes the data follow a distribution, usually normal, and tests a parameter such as a mean or variance. A non-parametric test assumes no particular distribution, which makes it more robust, but less powerful when the parametric assumptions hold.
When should a non-parametric test be used?
Use one for ranked, ordinal or categorical data, for fat-tailed or outlier-heavy data, for small samples, or whenever the central limit theorem cannot be relied on. When the distribution is known or the assumptions hold, the parametric test is better.
Which non-parametric tests are on the Level I exam?
In Quantitative Methods, the Spearman rank correlation test (for ranks or non-normal data) and the chi-square test of independence (for two categorical variables in a contingency table).
Spotted an error? Tell us and we will fix it.