Statistics · Topic 7 of 13
Linear Regression
Theory
Linear regression is a statistical method used to model the relationship between two numerical variables. We use it to draw a "line of best fit" through a scatterplot, which allows us to make predictions.
1. The Equation of the Regression Line
The equation of a straight line is typically written as y = mx + c or y = c + mx, where m is the slope (gradient) and c is the y-intercept.
- In R Studio, we generate this equation using the command
lm(Y~X)(which stands for linear model). - The R Studio output will provide two "Coefficients". The
(Intercept)is the y-intercept, and the value next to the independent variable is the slope.
2. Interpreting the Slope and Intercept (The Exam Rules)
When asked to interpret these parameters, you must relate them directly to the context of the data:
Slope:
"For every additional 1 [unit] of [Independent Variable], the [Dependent Variable] increases/decreases by [Slope value]"
Intercept:
"If the [Independent Variable] is 0, the estimated [Dependent Variable] would be [Intercept value]"
Note:
Sometimes the intercept is mathematically correct but physically impossible in real life (e.g., negative time, or a negative weight). You may be asked to point this out.
3. Making Predictions & Assessing Reliability
You can make predictions either by substituting an x-value into your equation, or by using the R Studio predict command: predict(lm(Y~X), newdata=data.frame(X=C), interval="pred").
- When reading the output of a predict command, the predicted value is always the number listed under fit.
- Reliability: A prediction is only reliable if it is an interpolation — meaning the value you are testing is strictly within the range of the data used to make the model, and the model has a strong correlation.
- If you make a prediction using a value outside the original data range, this is called extrapolation and is generally unreliable.
Worked examples
Example 1
Example 1: Extracting the Equation and Interpreting
A researcher is investigating the relationship between the number of hours of sunshine in a day and the number of ice creams sold at a local kiosk. They use statistical software and generate the following output:
Call:
lm(formula = Sales ~ Sunshine)
Coefficients:
(Intercept) Sunshine
15.4 22.8
- (a) State the equation of the regression line.
- (b) Interpret the slope and intercept parameters in the context of the data.
(a) Sales = 15.4 + 22.8 × Sunshine
(b)
- Slope: For every additional 1 hour of sunshine, the number of ice creams sold increases by 22.8.
- Intercept: On a day with 0 hours of sunshine, the estimated number of ice creams sold would be 15.4.
Example 2
Example 2: R Studio Predictions and Interpolation
A car manufacturer models the relationship between the weight of a car (in kg) and its fuel efficiency (in miles per gallon, mpg). The data used to build the model ranges from car weights of 900 kg to 1800 kg. There is a strong negative correlation between the variables.
The manufacturer uses R Studio to predict the fuel efficiency of a car that weighs 1400 kg:
> predict(lm(Efficiency~Weight),newdata=data.frame(Weight=1400),interval="pred")
fit lwr upr
1 45.32104 38.11542 52.52666
- (a) State the estimated fuel efficiency for a car weighing 1400 kg.
- (b) Comment on the reliability of this predicted value.
(a) 45.3 mpg
(The value is taken directly from the 'fit' column).
(b) The prediction is reliable because 1400 kg is within the range of the data used to make the model (900 kg to 1800 kg). It is an interpolation based on a strong linear model.
Example 3
Example 3: Extrapolation and Impossible Intercepts
A fitness instructor tracks the distance their clients run on a treadmill (in km) against the number of calories burned. The running distances recorded range from 2 km to 10 km. The equation of the regression line is found to be:
Calories = -35.2 + 65.5 × Distance
- (a) Estimate the number of calories burned by a client who runs 25 km, and comment on the accuracy of this prediction.
- (b) Use the intercept parameter to explain why this mathematical model is not entirely realistic.
(a) Calories = -35.2 + (65.5 × 25) = 1602.3 calories.
This prediction is unlikely to be accurate because 25 km is far outside the range of the observed data (2 km to 10 km). This is an extrapolation.
(b) The intercept is -35.2.
This means that if a client runs 0 km, they would burn -35.2 calories, which is physically impossible.