Outcomes and Type I and Type II Errors

7.3. Outcomes and Type I and Type II Errors#

In hypothesis testing, our decision to reject or fail to reject the null hypothesis (\(H_0\)) can lead to correct conclusions or errors. Understanding these potential outcomes is crucial for interpreting test results accurately and assessing the reliability of our statistical inferences.

7.3.1. The Four Possible Outcomes#

When conducting a hypothesis test, there are four possible scenarios based on the true state of the population and our decision:

  1. Correctly fail to reject a true \(H_0\) (True Negative)

  2. Incorrectly reject a true \(H_0\) (Type I error - False Positive)

  3. Correctly reject a false \(H_0\) (True Positive)

  4. Incorrectly fail to reject a false \(H_0\) (Type II error - False Negative)

These outcomes can be summarized in the following table:

Table 7.2 Outcomes in Hypothesis Testing#

Preliminary Conclusion

True State of Nature:
Null hypothesis is True

True State of Nature:
Null hypothesis is False

Fail to reject \(H_0\)

Correct decision (True Negative)

Type II Error: Fail to reject a false \(H_0\).
P(Type II error) = \(\beta\) (False Negative)

Reject \(H_0\)

Type I error: Reject a true \(H_0\).
P(Type I error) = \(\alpha\) (False Positive)

Correct decision (True Positive)

7.3.2. Type I and Type II Errors#

Let’s delve deeper into the two types of errors:

Definition - Type I Error and Significance Level

A Type I error occurs when we wrongly reject the null hypothesis when it is actually true. The probability of committing a Type I error is denoted by \(\alpha\) (alpha) and is called the significance level of the hypothesis test.

(7.1)#\[\begin{equation} \alpha = P(\text{Type I error}) = P(\text{rejecting } H_0 \text{ when } H_0 \text{ is true}) \end{equation}\]

Definition - Type II Error

A Type II error occurs when we do not reject the null hypothesis, despite it being false. The probability of committing a Type II error is denoted by \(\beta\) (beta).

(7.2)#\[\begin{equation} \beta = P(\text{Type II error}) = P(\text{failing to reject } H_0 \text{ when } H_0 \text{ is false}) \end{equation}\]

7.3.3. Relation between Type I and Type II Error Probabilities#

When the significance level (\(\alpha\)) is set to a smaller value for a fixed sample size, the probability of a Type II error (\(\beta\)) increases. In other words, reducing the significance level enhances the chances of retaining the null hypothesis when it is actually false.

7.3.4. Possible Conclusions for a Hypothesis Test#

Assuming a hypothesis test is performed with a small significance level:

  • If the null hypothesis is rejected, we conclude that the data offer substantial evidence to support the alternative hypothesis.

  • If the null hypothesis is not rejected, we conclude that the data do not provide enough evidence to support the alternative hypothesis.

Understanding these concepts helps researchers balance the risks in their hypothesis tests and interpret results with appropriate caution.

7.3.5. Examples to Illustrate Each Outcome#

Example 7.11 (True Negative (Correct Decision))

Scenario: A clinical trial tests if a new drug lowers blood pressure.

  • Null Hypothesis (\(H_0\)): The drug does not lower blood pressure.

  • Alternative Hypothesis (\(H_a\)): The drug lowers blood pressure.

Outcome: The trial shows no difference in blood pressure between the drug and a placebo. We do not reject the null hypothesis. Since the drug truly has no effect, this decision is correct (True Negative).

Explanation: This is a True Negative because we correctly concluded that the drug does not work when it actually doesn’t.

Example 7.12 (False Positive (Type I Error))

Scenario: A company tests if a new marketing strategy boosts sales.

  • Null Hypothesis (\(H_0\)): The strategy does not increase sales.

  • Alternative Hypothesis (\(H_a\)): The strategy increases sales.

Outcome: Sales data suggests an increase, so we reject the null hypothesis. However, the strategy does not actually boost sales. This is a Type I error (False Positive).

Explanation: This is a False Positive because we incorrectly concluded that the strategy works when it doesn’t.

Example 7.13 (True Positive (Power of the Test))

Scenario: A researcher tests if a new diet lowers cholesterol.

  • Null Hypothesis (\(H_0\)): The diet does not lower cholesterol.

  • Alternative Hypothesis (\(H_a\)): The diet lowers cholesterol.

Outcome: Data shows cholesterol reduction with the diet, so we reject the null hypothesis. The diet truly lowers cholesterol, making this decision correct (True Positive).

Explanation: This is a True Positive because we correctly concluded that the diet works when it actually does.

Example 7.14 (False Negative (Type II Error))

Scenario: An agriculturist tests if a new fertilizer increases crop yield.

  • Null Hypothesis (\(H_0\)): The fertilizer does not increase yield.

  • Alternative Hypothesis (\(H_a\)): The fertilizer increases yield.

Outcome: Data shows no significant yield increase, so we do not reject the null hypothesis. However, the fertilizer actually does increase yield, leading to a Type II error (False Negative).

Explanation: This is a False Negative because we incorrectly concluded that the fertilizer doesn’t work when it actually does.