Statistics · Topic 13 of 13
Mixed Hypothesis Tests
Theory
In the real world, and specifically in your final Qualifications Scotland exam and project, you will not always be told which statistical test to run. You must be able to analyse the research question and the type of data provided to choose the correct test yourself.
1. The Decision Tree for Choosing a Test
To select the correct test, ask yourself what type of data you are analysing and what you are trying to find out:
Are you comparing two proportions or percentages?
- Data Type: Categorical (e.g., Pass/Fail, Yes/No).
- Test to use: Z-test (2-sample test for equality of proportions).
- Hypothesis Keyword: "Proportion".
Are you looking for a relationship between two numerical variables?
- Data Type: Two numerical variables measured on the same subjects (e.g., Height and Weight).
- Test to use: Correlation Test.
- Hypothesis Keyword: "Relationship".
Are you comparing the averages of two completely separate groups?
- Data Type: Numerical data, normally distributed, from two independent groups (e.g., Group A vs Group B).
- Test to use: Independent t-test.
- Hypothesis Keyword: "Mean".
Are you comparing the averages of the exact same group measured twice?
- Data Type: Numerical data, normally distributed, from the same subjects (e.g., Before and After).
- Test to use: Paired t-test.
- Hypothesis Keyword: "Mean".
2. Forming Conclusions (A Quick Recap)
Regardless of which test you choose, the rules for interpreting the R Studio output remain exactly the same:
- p < 0.05: The result is statistically significant. You must reject the null hypothesis.
- p ≥ 0.05: The result is not statistically significant. You must fail to reject the null hypothesis.
- Confidence Intervals: If the 95% confidence interval contains zero, zero is a plausible value for the difference, which is consistent with failing to reject the null hypothesis. If it does not contain zero, it provides further evidence to reject the null hypothesis.
Worked examples
Example 1
Example 1: Identifying the Correct Test
A researcher is designing four different studies. State the most appropriate statistical hypothesis test for each study:
- (a) Investigating if there is a difference in the mean reaction times between professional athletes and amateur athletes.
- (b) Investigating if there is a relationship between the number of hours a person spends sleeping and their concentration score.
- (c) Investigating if there is a difference in the proportion of electric cars driven in Edinburgh compared to Glasgow.
- (d) Investigating if there is a difference in the mean blood pressure of a group of patients before they take a new medication and one hour after they take it.
(a) Independent t-test (Comparing the means of two completely separate groups).
(b) Correlation test (Looking for a relationship between two numerical variables).
(c) Z-test / 2-sample test for equality of proportions (Comparing proportions from categorical data).
(d) Paired t-test (Comparing the means of the exact same group measured twice).
Example 2
Example 2: Formulating Hypotheses from a Scenario
A marine biologist has gathered categorical data on the number of male and female sea turtles that successfully migrate to a specific island. They want to know if gender affects migration success.
- (a) State the statistical test the biologist should run.
- (b) State the appropriate null (H0) and alternative (H1) hypotheses.
(a) A z-test (or a 2-sample test for equality of proportions).
(b)
H0: There is no difference in the proportion of male and female sea turtles that successfully migrate.
H1: There is a difference in the proportion of male and female sea turtles that successfully migrate.
Example 3
Example 3: Interpreting a Mixed Test Output
A psychologist wants to find out if listening to classical music changes students' test scores. They give 50 students a memory test in silence. The next day, they give the exact same 50 students a similar memory test while classical music plays. The data is found to be normally distributed.
The psychologist runs the appropriate test in R Studio and receives a p-value of 0.003 and a 95% confidence interval of [-12.4, -4.2].
- (a) State the name of the statistical test the psychologist ran.
- (b) Interpret the p-value and the result of the hypothesis test in context.
- (c) Explain how the confidence interval supports this conclusion.
(a) A paired t-test (Because the numerical data is normally distributed and comes from the same group tested twice).
(b) Since the p-value (0.003) is less than 0.05, we reject the null hypothesis. There is statistically significant evidence to suggest that there is a difference in the mean memory test scores of students when listening to classical music compared to silence.
(c) We can be 95% confident that the true difference in the mean scores lies between -12.4 and -4.2. Because this interval does not contain zero, it provides further statistical evidence that a true difference in the means exists.