You are reading the 2026 curriculum (exams up to November 2026). Sitting in February 2027 or later? Go to the 2027 Quantitative Methods guide.

Learning Module 10 of Quantitative Methods 6–9% of the exam

Simple Linear Regression

6 lessons. About 40 minutes in cram mode, 200 minutes in full.

Start with lesson 1 Learn with Sensei Read the module summary

Lessons

1
The regression line: slope, intercept and least squares
describe 8 min cram, 36 min full
2
The four regression assumptions and what residual plots reveal
explain 6 min cram, 30 min full
3
Goodness of fit and testing the regression coefficients
calculate 8 min cram, 40 min full
4
The ANOVA table and the standard error of estimate
calculate 6 min cram, 30 min full
5
Predicting Y and building a prediction interval
calculate 6 min cram, 30 min full
6
Functional forms: log-lin, lin-log and log-log models
describe 6 min cram, 34 min full

What the exam expects you to do

Describe a simple linear regression model, how the least squares criterion is used to estimate regression coefficients, and the interpretation of these coefficients.The regression line: slope, intercept and least squares
Explain the assumptions underlying the simple linear regression model, and describe how residuals and residual plots indicate if these assumptions may have been violated.The four regression assumptions and what residual plots reveal
Calculate and interpret measures of fit and formulate and evaluate tests of fit and of regression coefficients in a simple linear regression.Goodness of fit and testing the regression coefficients
Describe the use of analysis of variance (ANOVA) in regression analysis, interpret ANOVA results, and calculate and interpret the standard error of estimate in a simple linear regression.The ANOVA table and the standard error of estimate
Calculate and interpret the predicted value for the dependent variable, and a prediction interval for it, given an estimated linear regression model and a value for the independent variable.Predicting Y and building a prediction interval
Describe different functional forms of simple linear regressions.Functional forms: log-lin, lin-log and log-log models

Module summary

The whole module on one sheet, organised by lesson: what to memorize, how the ideas fit together, the formulas and the key terms. No paragraphs, just what you need to revise.

Hide what I know

How deeply to learn each part

Know the idea: Understand it in general. It frames other topics, but you will not calculate with it.
Use it: Understand it well enough to solve a question with it. Recognize when it applies.
Memorize: Learn it by heart. The exam will not give it to you.

The regression line: slope, intercept and least squares

Known

MemorizeThe least squares line

Item	What to know
Model	$Y_i = b_0 + b_1X_i + \varepsilon_i$
Slope	$b_1 = \text{Cov}(X,Y) \div \text{Var}(X)$
Intercept	$b_0 = \bar{Y} - b_1\bar{X}$
Residual	$e_i = Y_i - \hat{Y}_i$ (actual − fitted)
Criterion	minimize $\sum e_i^2$ (SSE)

Memory hook Slope = covariance over the variance of X, the variable you stand on. Divide by both standard deviations and you get r, not the slope. The line always passes through (X̄, Ȳ), so the intercept just finishes the job.

Know it cold Least squares minimizes the sum of SQUARED residuals. Plain residuals always sum to zero, so they cannot be the criterion.

Key terms

simple linear regression Use it
dependent variable Memorize
independent variable Memorize
regression coefficients Use it
intercept Memorize
slope coefficient Memorize
error term Use it
residual Memorize
fitted value Use it
sum of squares error Memorize
least squares Memorize

The parts of a simple linear regression

Y = b₀ + b₁X + εone explained variable, one explaining variable

Dependent variable, Y Memorize the variable you want to explain
- Also called explained variable, predicted variable
Independent variable, X Memorize the variable doing the explaining
- Also called explanatory variable, predictor
Regression coefficients Memorize estimated from the data
- Intercept, b₀ predicted Y when X = 0
- Slope, b₁ change in predicted Y per one-unit rise in X
Error term, ε Use it what the line leaves unexplained
- Residual, e the sample estimate of ε: actual Y minus fitted Y

Formulas

Simple linear regression model Memorize

Y_i = b_0 + b_1 X_i + \varepsilon_i

Quick example

Revenue growth (Y, %) on marketing spend (X, % of sales): Y = 1.39 + 1.29X + ε. A retailer spending 4% of sales is fitted at 1.39 + 1.29 × 4 = 6.55%; the gap to its actual growth is its residual.

On your calculator

1.29×4+1.39= 6.55

1.29ENTER4×1.39+ 6.55

ENTER enter (on the HP, puts the number on the stack)

Slope coefficient Memorize

b_1 = \dfrac{\sum (X_i-\bar{X})(Y_i-\bar{Y})}{\sum (X_i-\bar{X})^2} = \dfrac{\text{Cov}(X,Y)}{\text{Var}(X)}

Quick example

Cov(X, Y) = 0.0018 and Var(X) = 0.0024. Slope = 0.0018 ÷ 0.0024 = 0.75.

On your calculator

0.0018÷0.0024= 0.75

0.0018ENTER0.0024÷ 0.75

ENTER enter (on the HP, puts the number on the stack)

Intercept Memorize

b_0 = \bar{Y} - b_1\bar{X}

Quick example

Ȳ = 5, X̄ = 4 and slope 0.75. Intercept = 5 − 0.75 × 4 = 2.00.

On your calculator

5−(0.75×4)= 2

5ENTER0.75ENTER4×− 2

ENTER enter (on the HP, puts the number on the stack)

Slope from the correlation Use it

b_1 = r\,\dfrac{s_Y}{s_X}

Quick example

Correlation 0.60, standard deviation of Y 5%, of X 4%. Slope = 0.60 × 5 ÷ 4 = 0.75.

On your calculator

0.6×5÷4= 0.75

0.6ENTER5×4÷ 0.75

ENTER enter (on the HP, puts the number on the stack)

The four regression assumptions and what residual plots reveal

Known

MemorizeAssumption, tell-tale, example

Assumption	Residual plot when broken	Typical cause
Linearity	U-shape or arch	compounding series fitted with a straight line
Homoskedasticity	fan or funnel; step change in spread	spread widens as X grows; two regimes pooled
Independence	runs of + then −, seasonal waves	time series with trends or seasonality
Normality	skewed or fat-tailed histogram, outliers	a data-entry error, extreme events

Memory hook Four assumptions, four shapes: curve, fan, wave, lopsided. Good residuals look like static on a TV: a level band with no shape.

Know it cold The assumptions are about the errors and the model (linear in the parameters), not about X being normal. A high R² does not prove them.

Key terms

linearity Memorize
homoskedasticity Memorize
heteroskedasticity Memorize
independence of errors Memorize
serial correlation Use it
normality of errors Memorize
residual plot Use it
outlier Use it
regime Know the idea

Assumptions and how residual plots expose a breach

Simple linear regression assumptionscheck them with the residuals

Linearity Memorize Y is a linear function of X in the parameters
- Broken: curve or arch in the residuals fix: transform the variables
Homoskedasticity Memorize error variance is the same for every observation
- Broken (heteroskedasticity): fan, funnel or step change fix: split the sample or transform
Independence of errors Memorize one error tells you nothing about the next
- Broken (serial correlation): runs, waves, seasonal pattern common in time series
Normality of errors Memorize matters most in small samples
- Broken: skewed or fat-tailed histogram, outliers check for data errors first

Goodness of fit and testing the regression coefficients

Known

MemorizeFit and coefficient tests

Item	Rule
$R^2$	$SSR \div SST$ (explained share); equals $r^2$ in simple regression
t-statistic	$(b - \text{hypothesized}) \div s_b$
Degrees of freedom (t)	$n - 2$
F-statistic	$MSR \div MSE$ , df $1$ and $n-2$ ; $F = t^2$ for the slope
Decision	$\|t\| >$ critical → reject; or reject when $p <$ $\alpha$
Indicator X	intercept = mean of the 0 group; slope = difference in means

Memory hook The slope test is "estimate minus claim, over its own standard error": the same shape as every t-test you know. n − 2 because two coefficients were estimated.

Know it cold To test a slope against 1, subtract 1 first. Testing against zero is the special case where there is nothing to subtract.

Key terms

coefficient of determination Memorize
F-test of the model Use it
standard error of the slope coefficient Use it
t-test of the slope Memorize
test of the intercept Use it
indicator variable Use it
p-value Use it

Judging a simple regression

Is the regression any good?fit first, then tests

Goodness of fit Memorize how much of Y the line explains
- R² = SSR ÷ SST unit-free share, 0 to 1
- F-test of the model H₀: slope is zero; one-tailed, df 1 and n − 2
Tests of the coefficients Memorize t-tests with n − 2 degrees of freedom
- Slope t = (b₁ − B₁) ÷ s_b₁
- Intercept t = (b₀ − B₀) ÷ s_b₀
Reading the result Use it compare with the critical value or p-value
- |t| above critical value, or p below α reject H₀: the coefficient differs from the hypothesized value
- |t| at or below critical value fail to reject H₀

Formulas

Coefficient of determination Memorize

R^2 = \dfrac{SSR}{SST} = 1 - \dfrac{SSE}{SST}

Quick example

SST = 120 and SSE = 30, so SSR = 90. R² = 90 ÷ 120 = 0.75: the model explains 75% of the variation in Y.

On your calculator

120−30=÷120= 0.75

120ENTER30−120÷ 0.75

ENTER enter (on the HP, puts the number on the stack)

Standard error of the slope coefficient Use it

s_{b_1} = \dfrac{SEE}{\sqrt{\sum (X_i-\bar{X})^2}}

Quick example

SEE = 2.0 and Σ(X − X̄)² = 64. s_b₁ = 2.0 ÷ √64 = 2.0 ÷ 8 = 0.25.

On your calculator

2÷64√x= 0.25

√x square root

2ENTER64g√x÷ 0.25

ENTER enter (on the HP, puts the number on the stack) · g blue shift key · √x square root

t-statistic for a regression coefficient Memorize

t = \dfrac{b_1 - B_1}{s_{b_1}},\quad df = n-2

Quick example

Slope 1.35, standard error 0.20, testing H₀: B₁ = 1. t = (1.35 − 1) ÷ 0.20 = 1.75. With a critical value near 2.0 you fail to reject.

On your calculator

1.35−1=÷0.2= 1.75

1.35ENTER1−0.2÷ 1.75

ENTER enter (on the HP, puts the number on the stack)

F equals t squared (simple regression) Use it

F = t_{b_1}^{\,2}

Quick example

The slope has t = 3.0 against zero, so the F-statistic is 3.0² = 9.0.

On your calculator

3x²= 9

x² square

3ENTER× 9

ENTER enter (on the HP, puts the number on the stack)

The ANOVA table and the standard error of estimate

Known

MemorizeThe ANOVA table

Source	Sum of squares	df	Mean square	F
Regression	SSR	$1$	$MSR = SSR \div 1$	$MSR \div MSE$
Error	SSE	$n-2$	$MSE = SSE \div (n-2)$
Total	SST = SSR + SSE	$n-1$

Memory hook Degrees of freedom add up: 1 + (n − 2) = n − 1. Sums of squares add up too. Mean squares do not.

Know it cold $SEE = \sqrt{MSE}$ (root of SSE ÷ (n − 2)); $R^2 = SSR \div SST$ ; $F = MSR \div MSE$ . Three numbers and n unlock the whole table.

Key terms

analysis of variance (ANOVA) Use it
sum of squares total Memorize
sum of squares regression Memorize
mean square Use it
mean square error Memorize
standard error of estimate Memorize
F-statistic Memorize

Where each ANOVA number comes from

Total variation, SSTdf = n − 1

Explained by the line: SSR Memorize df = 1 in simple regression
- MSR = SSR ÷ 1 the numerator of F
- R² = SSR ÷ SST share explained
Left unexplained: SSE Memorize df = n − 2
- MSE = SSE ÷ (n − 2) the denominator of F
- SEE = √MSE typical size of a residual, in the units of Y

Formulas

Decomposition of variation Memorize

SST = SSR + SSE

Quick example

SSR = 36 and SSE = 50 give SST = 86. Degrees of freedom split the same way: n − 1 = 1 + (n − 2).

Mean square error Memorize

MSE = \dfrac{SSE}{n-2}

Quick example

n = 27 and SSE = 50. MSE = 50 ÷ 25 = 2.00.

On your calculator

50÷(27−2)= 2

50ENTER27ENTER2−÷ 2

ENTER enter (on the HP, puts the number on the stack)

Standard error of estimate Memorize

SEE = \sqrt{MSE} = \sqrt{\dfrac{SSE}{n-2}}

Quick example

MSE = 2.00, so SEE = √2.00 = 1.41, in the units of Y.

On your calculator

50÷(27−2)=√x 1.4142

√x square root

50ENTER27ENTER2−÷g√x 1.4142

ENTER enter (on the HP, puts the number on the stack) · g blue shift key · √x square root

F-statistic Memorize

F = \dfrac{MSR}{MSE} = \dfrac{SSR \,/\, 1}{SSE \,/\, (n-2)}

Quick example

SSR = 36, SSE = 50, n = 27. MSR = 36, MSE = 2.00, so F = 36 ÷ 2.00 = 18.0.

On your calculator

36÷(50÷25)= 18

36ENTER50ENTER25÷÷ 18

ENTER enter (on the HP, puts the number on the stack)

Predicting Y and building a prediction interval

Known

MemorizePrediction recipe

Step	Rule
Point forecast	$\hat{Y} = b_0 + b_1X_f$
Interval	$\hat{Y} \pm t_c \times s_f$ , $df = n-2$
$s_f$ vs SEE	$s_f > SEE$ always; $s_f^2 = SEE^2[1 + 1/n + (X_f-\bar{X})^2/\sum(X_i-\bar{X})^2]$
Narrowest at	$X_f = \bar{X}$ ; wider as $X_f$ moves away

Memory hook Three sources of doubt, three terms under the root: the new error itself (the 1), the line being estimated from a finite sample (1/n), and the slope error magnified away from the middle (the distance term).

Know it cold Use the t critical value with n − 2 degrees of freedom, not z, and s_f, not SEE.

Key terms

predicted value Memorize
prediction interval Memorize
standard error of the forecast Use it
critical t-value Use it

What makes a prediction interval wider

Width of Ŷ ± t_c × s_fa bigger s_f or t_c means a wider interval

Larger SEE Use it noisier data around the line
Smaller sample, n Use it the 1/n term and the larger t_c
X_f farther from X̄ Memorize the distance term grows
Higher confidence level Use it a larger t_c

Formulas

Predicted value Memorize

\hat{Y}_f = b_0 + b_1 X_f

Quick example

Intercept 2.0, slope 0.75, forecast X = 6. Ŷ = 2.0 + 0.75 × 6 = 6.5.

On your calculator

0.75×6+2= 6.5

0.75ENTER6×2+ 6.5

ENTER enter (on the HP, puts the number on the stack)

Standard error of the forecast Use it

s_f = SEE\sqrt{1 + \dfrac{1}{n} + \dfrac{(X_f-\bar{X})^2}{\sum (X_i-\bar{X})^2}}

Quick example

SEE = 2.0, n = 20 and X_f = X̄. s_f = 2.0 × √(1 + 1/20) = 2.05, just above the SEE.

On your calculator

2×(1+201/x)√x= 2.0494

1/x reciprocal · √x square root

201/x1+g√x2× 2.0494

1/x reciprocal · g blue shift key · √x square root

Prediction interval Memorize

\hat{Y}_f \pm t_c \times s_f,\quad df = n-2

Quick example

Ŷ = 5.5, t_c = 2.05, s_f = 1.2. Interval = 5.5 ± 2.05 × 1.2 = 5.5 ± 2.46 = 3.04 to 7.96.

On your calculator

2.05×1.2=STO1 2.46
5.5−RCL1= 3.04
5.5+RCL1= 7.96

STO store · RCL recall a stored number

2.05ENTER1.2×STO1 2.46
5.5RCL1− 3.04
5.5RCL1+ 7.96

ENTER enter (on the HP, puts the number on the stack) · STO store · RCL recall a stored number

Functional forms: log-lin, lin-log and log-log models

Known

MemorizeThree functional forms

Model	Equation	Slope means	Fits
Log-lin	$\ln Y = b_0 + b_1X$	relative change in Y per unit X ( $\approx b_1 \times 100\%$ )	steady percentage growth
Lin-log	$Y = b_0 + b_1\ln X$	change in Y for a relative change in X (1% gives $b_1 \times 0.01$ )	effects that fade as X grows
Log-log	$\ln Y = b_0 + b_1\ln X$	elasticity: % change in Y per 1% change in X	constant-elasticity relations

Memory hook Read the model name as "Y-side, X-side": the log sits on the side that is measured in percent. Log on Y means Y moves in percent; log on X means X moves in percent. Log on both: percent per percent.

Know it cold A prediction from a model with ln Y must be exponentiated: Y = e^(predicted ln Y). Compare R² only between models with the same dependent variable.

Key terms

functional form Use it
natural logarithm transformation Use it
log-lin model Memorize
lin-log model Memorize
log-log model Memorize
elasticity Memorize

Choosing a functional form

Which side of the equation is in logs?logs straighten a curved scatter

Neither: lin-lin Use it the ordinary straight-line model
Y only: log-lin Memorize ln Y = b₀ + b₁X
- Use for constant percentage growth over time
- Slope relative change in Y per unit X
X only: lin-log Memorize Y = b₀ + b₁ ln X
- Use for diminishing effects
- Slope change in Y per relative change in X
Both: log-log Memorize ln Y = b₀ + b₁ ln X
- Use for constant-elasticity relations
- Slope elasticity, % change in Y per 1% in X

Formulas

Log-lin model Memorize

\ln Y_i = b_0 + b_1 X_i + \varepsilon_i

Quick example

ln Y = 1.0 + 0.10X. At X = 3, ln Y = 1.30, so Y = e^1.30 = 3.67. The slope says Y grows about 10% per unit of X.

On your calculator

0.1×3+1=2NDe^x 3.6693

2ND second function (the shifted label above a key) · e^x e to the power x

0.1ENTER3×1+ge^x 3.6693

ENTER enter (on the HP, puts the number on the stack) · g blue shift key · e^x e to the power x

Lin-log model Memorize

Y_i = b_0 + b_1 \ln X_i + \varepsilon_i

Quick example

Y = 4 + 2 ln X. X rises from 50 to 60. Y changes by 2 × ln(60 ÷ 50) = 2 × 0.1823 = 0.36.

On your calculator

60÷50=LN×2= 0.3646

LN natural logarithm

60ENTER50÷gLN2× 0.3646

ENTER enter (on the HP, puts the number on the stack) · g blue shift key · LN natural logarithm

Log-log model Memorize

\ln Y_i = b_0 + b_1 \ln X_i + \varepsilon_i

Quick example

Price elasticity of demand b₁ = −1.2. A 5% price rise goes with a change in quantity of about −1.2 × 5% = −6%.

On your calculator

1.2+|-×5= −6

+|- change sign

1.2CHSENTER5× −6

CHS change sign · ENTER enter (on the HP, puts the number on the stack)

Converting a log-lin prediction back to Y Memorize

Y = e^{\,b_0 + b_1 X}

Quick example

ln Y = 2.0 + 0.05X. At X = 20: ln Y = 3.0, so Y = e^3.0 = 20.09.

On your calculator

0.05×20+2=2NDe^x 20.0855

2ND second function (the shifted label above a key) · e^x e to the power x

0.05ENTER20×2+ge^x 20.0855

ENTER enter (on the HP, puts the number on the stack) · g blue shift key · e^x e to the power x

Cheat sheet · PDF · 4 A4 pages

Simple Linear Regression on 4 pages

What to memorize, the concept maps, every formula with a worked example and keystrokes for the TI BA II Plus and HP 12C, how each idea is asked, and the traps. Print it, or keep it on your phone.

View Download PDF

Free with an account. You come straight back here after signing up.