3.6. Conditional Probability

3.6.1. Conditional Probability

Definition - Conditional Probability

Conditional probability is the likelihood of an event occurring, given that another specific event has already occurred. In probability theory, the conditional probability of an event \(A\) given event \(B\) is denoted as \( P(A|B) \). This measures the probability of event \(A\) when event \(B\) is known to have occurred, effectively reducing the sample space to only those outcomes where event \(B\) is true [Shankar, 2021].

Mathematically, the conditional probability \( P(A|B) \) is defined by the formula:

(3.8)\[\begin{equation} P(A|B) = \dfrac{P(A \cap B)}{P(B)} \end{equation}\]

Here:

• \( P(A \cap B) \) represents the probability of both events \(A\) and \(B\) occurring simultaneously, also known as the intersection of \(A\) and \(B\).

• \( P(B) \) is the probability of event B occurring.

Example 3.28 (Weather and Traffic)

A city tracks its weather and traffic conditions. Over the course of a year, they gathered the following data:

• There were 80 rainy days.

• There were 120 days with heavy traffic.

• There were 60 days that were both rainy and had heavy traffic.

Let:

• A: The event that a day has heavy traffic

• B: The event that a day is rainy

Calculate the probability of heavy traffic given that it’s a rainy day, P(A|B).

Solution: To solve this, we’ll use the definition of conditional probability:

\[\begin{equation*} P(A|B) = \frac{P(A \cap B)}{P(B)} \end{equation*}\]

Let’s calculate each component:

1. P(B): The probability of a rainy day

   \[\begin{equation*} P(B) = \frac{\text{Number of rainy days}}{\text{Total days}} = \frac{80}{365} = \frac{16}{73} \approx 0.2192 \end{equation*}\]

2. P(A ∩ B): The probability of a day being both rainy and having heavy traffic

   \[\begin{equation*} P(A \cap B) = \frac{\text{Number of rainy days with heavy traffic}}{\text{Total days}} = \frac{60}{365} = \frac{12}{73} \approx 0.1644 \end{equation*}\]

Now, we can substitute these values into our conditional probability formula:

\[\begin{equation*} P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{12/73}{16/73} = \frac{12}{16} = \frac{3}{4} \approx 0.7500 \end{equation*}\]

Therefore, the probability of heavy traffic given that it’s a rainy day is 3/4 or 75%.

Example 3.29 (College Admissions)

A college admissions office has the following data about applicants:

• 500 students applied in total

• 300 students had high test scores

• 200 students were admitted

• 180 students with high test scores were admitted

Let:

• A: The event that a student is admitted

• B: The event that a student has high test scores

Calculate the probability of a student being admitted given that they have high test scores, P(A|B).

Solution: We’ll use the definition of conditional probability:

\[\begin{equation*} P(A|B) = \frac{P(A \cap B)}{P(B)} \end{equation*}\]

Let’s calculate each component:

1. \(P(B)\): The probability of a student having high test scores

   \[\begin{equation*} P(B) = \frac{\text{Students with high test scores}}{\text{Total applicants}} = \frac{300}{500} = \frac{3}{5} \approx 0.6000 \end{equation*}\]

2. \(P(A \cap B)\): The probability of a student being admitted and having high test scores

   \[\begin{equation*} P(A \cap B) = \frac{\text{Admitted students with high test scores}}{\text{Total applicants}} = \frac{180}{500} = \frac{9}{25} \approx 0.3600 \end{equation*}\]

Now, we can calculate the conditional probability:

\[\begin{equation*} P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{9/25}{3/5} = \frac{9/25}{3/5} \cdot \frac{5}{5} = \frac{3}{5} \approx 0.6000 \end{equation*}\]

Therefore, the probability of a student being admitted given that they have high test scores is 3/5 or 60%.

3.6.2. Bayes’ Theorem

Bayes’ Theorem is a fundamental result in probability theory that provides a way to update the probability of an event, \(A\), based on new evidence, \(B\). The theorem is expressed as follows [Koch, 2007, Navidi and of Mines, 2018]:

(3.9)\[\begin{equation} P(A|B) = \dfrac{P(B|A) \cdot P(A)}{P(B)}. \end{equation}\]

3.6.2.1. Key Probabilities in Bayes’ Theorem

The formula incorporates several important components:

• Prior probability (\(P(A)\)): This represents the initial probability of event \(A\) before any new information is taken into account.

• Likelihood (\(P(B|A)\)): This quantifies the probability of observing the evidence, \(B\), assuming that event \(A\) is true. It reflects how consistent the new evidence is with the hypothesis.

• Marginal probability (\(P(B)\)): This is the total probability of observing event \(B\), accounting for all possible causes. It serves as a normalizing factor to ensure that the probabilities sum to 1.

• Posterior probability (\(P(A|B)\)): This is the updated probability of event \(A\) after observing the evidence \(B\). It represents a revision of the prior belief in light of the new data.

3.6.2.2. Derivation of Bayes’ Theorem

To derive Bayes’ Theorem, we begin with the definition of conditional probability. The probability of \(A\) given \(B\), denoted by \(P(A|B)\), is defined as the ratio of the joint probability of both \(A\) and \(B\) occurring to the probability of \(B\):

(3.10)\[\begin{equation} P(A|B) = \dfrac{P(A \cap B)}{P(B)}. \end{equation}\]

Similarly, the conditional probability of \(B\) given \(A\), denoted by \(P(B|A)\), is:

(3.11)\[\begin{equation} P(B|A) = \dfrac{P(A \cap B)}{P(A)}. \end{equation}\]

From this, the joint probability \(P(A \cap B)\) can be expressed in two equivalent ways:

(3.12)\[\begin{equation} P(A \cap B) = P(A) \cdot P(B|A), \end{equation}\]

or:

(3.13)\[\begin{equation} P(A \cap B) = P(B) \cdot P(A|B). \end{equation}\]

By substituting \(P(A \cap B) = P(A) \cdot P(B|A)\) into the definition of \(P(A|B)\), we get:

(3.14)\[\begin{equation} P(A|B) = \dfrac{P(A) \cdot P(B|A)}{P(B)}. \end{equation}\]

This is the formula for Bayes’ Theorem.

3.6.2.3. Application of Bayes’ Theorem

Bayes’ Theorem is instrumental in fields such as statistical inference, decision theory, and machine learning. It enables the updating of probabilities as new information becomes available. The theorem allows for a shift from initial beliefs (the prior probability) to updated beliefs (the posterior probability) in light of new evidence.

For example, in medical diagnostics, \(A\) could represent the probability of a patient having a certain disease, and \(B\) could represent the result of a medical test. Bayes’ Theorem would allow us to update the likelihood of the patient having the disease based on the test result.

3.6.2.4. Conditional Probabilities and Joint Probabilities

In the broader context of probability theory, the relationship between conditional and joint probabilities is fundamental. For instance:

• The probability of \(B\) given \(A\), denoted as \(P(B|A)\), can be computed from the joint probability \(P(A \cap B)\) divided by \(P(A)\):

  (3.15)\[\begin{equation} P(B|A) = \dfrac{P(A \cap B)}{P(A)}. \end{equation}\]

• Similarly, the probability of \(A\) given \(B\), denoted as \(P(A|B)\), can be derived from the joint probability \(P(A \cap B)\) divided by \(P(B)\):

  (3.16)\[\begin{equation} P(A|B) = \dfrac{P(A \cap B)}{P(B)}. \end{equation}\]

Thus, Bayes’ Theorem effectively connects the joint and conditional probabilities in a manner that allows for the iterative refinement of beliefs as new information is integrated.

3.6.3. Multiplication Rule

The probability of both event A and event B occurring together, denoted as \(P(A \text{ and } B) \), is calculated using the formula:

(3.17)\[\begin{equation} P(A \text{ and } B) = P(A) \times P(B | A) \end{equation}\]

Here, \( P(A) \) is the probability of event A occurring, and \( P(B | A) \) is the conditional probability of event B occurring given that event A has already occurred.

• The conditional probability of event B occurring given that event A has already occurred, represented as \(P(B | A)\), is determined by the formula:

  (3.18)\[\begin{equation} P(B | A) = \dfrac{P(A \text{ and } B)}{P(A)} \end{equation}\]

  This formula shows that the conditional probability \( P(B | A) \) is the ratio of the joint probability of A and B occurring together to the probability of A occurring on its own.

Example 3.30 (Conditional Probability in Card Drawing)

From a standard 52-card deck, we draw two cards sequentially without replacement. Let:

• A: First card is a heart

• B: Second card is a spade

Fig. 3.47 Standard playing cards. Source: Open Source Vector Playing Cards

Calculate:

• a. P(A): Probability of drawing a heart first

• b. P(B|A): Conditional probability of drawing a spade second, given the first card was a heart

Solution:

a. P(A): Probability of drawing a heart first

This is a simple probability calculation:

(3.19)\[\begin{equation} P(A) = \frac{\text{Number of hearts}}{\text{Total number of cards}} = \frac{13}{52} = \frac{1}{4} = 0.25 \end{equation}\]

b. P(B|A): Conditional probability of drawing a spade second, given the first card was a heart

After drawing a heart, we need to consider the new composition of the deck:

• Total cards left: 51

• Number of spades: 13 (unchanged, as we drew a heart)

Therefore, the probability of drawing a spade, given that we’ve already drawn a heart, is:

(3.20)\[\begin{equation} P(B|A) = \frac{\text{Number of spades}}{\text{Total cards left}} = \frac{13}{51} \approx 0.2549 \end{equation}\]

This result shows that if we know the first card is a heart, the probability of drawing a spade second is slightly higher than the initial probability of drawing a spade (which would be 13/52 = 0.25). This is because removing a heart from the deck slightly increases the proportion of spades in the remaining cards.

Note

This problem demonstrates the concept of dependent events. The probability of drawing a spade changes after we’ve drawn a heart because the composition of the deck has changed. In this case, we can directly calculate the new probability by considering the remaining cards in the deck.

Example 3.31 (Colored Marble Draw)

Consider a bag containing colored marbles:

• 4 red marbles

• 6 blue marbles

Fig. 3.48 An image of 4 red marbles and 6 blue marbles in a bag.

Let:

• A: Event of drawing a red marble

• B: Event of drawing a blue marble

Find the probability of both events occurring together (drawing a red marble first and then a blue marble).

Solution: To find the probability of both events occurring together, we calculate the probability of drawing a red marble first and multiply it by the probability of drawing a blue marble from the remaining marbles.

Red:

4 Marbles

Blue:

6 Marbles

1. Probability of drawing a red marble first:

   \[\begin{equation*} P(A) = \frac{\text{Number of red marbles}}{\text{Total number of marbles}} = \frac{4}{10} = \frac{2}{5} \end{equation*}\]

2. Probability of drawing a blue marble after a red one:

   \[\begin{equation*} P(B|A) = \frac{\text{Number of blue marbles}}{\text{Total remaining marbles}} = \frac{6}{9} = \frac{2}{3} \end{equation*}\]

3. The combined probability (joint probability):

   \[\begin{align*} P(A \cap B) &= P(A) \times P(B|A) \\ &= \frac{2}{5} \times \frac{2}{3} \\ &= \frac{4}{15} \\ &\approx 0.2667 \text{ or } 26.67\% \end{align*}\]

Therefore, the probability of drawing a red marble first and then a blue marble is 4/15 or approximately 0.2667 or 26.67%.

Example 3.32

Imagine you are playing with a standard deck of 52 cards. You draw cards one by one without replacement. Event A is drawing a heart, and event B is drawing a face card (Jack, Queen, or King). You want to find the probability of drawing a face card after already having drawn a heart.

Fig. 3.49 A standard deck of playing cards. Source: Open Source Vector Playing Cards

Questions:

• a. What is the probability of drawing a heart from the deck?

• b. After drawing a heart and not replacing it, what is the probability of then drawing a face card?

• c. What is the combined probability of first drawing a heart and then drawing a face card?

Solution:

a. Probability of Drawing a Heart (Event A): There are 13 hearts in a deck of 52 cards, so the probability of drawing a heart is:

\[\begin{equation*} P(A) = \frac{13}{52} = \frac{1}{4} = 0.25 \text{ or } 25\%\end{equation*}\]

b. Conditional Probability of Drawing a Face Card After a Heart (Event B | Event A): After drawing a heart, we need to consider two scenarios:

1. If the heart drawn was not a face card (10 out of 13 hearts):

   • 51 cards left, 12 face cards remain

2. If the heart drawn was a face card (3 out of 13 hearts):

   • 51 cards left, 11 face cards remain

We can calculate this as a weighted average:

\[\begin{align*} P(B|A) &= P(\text{Not face card}) \cdot \frac{12}{51} + P(\text{Face card}) \cdot \frac{11}{51} \\ &= \frac{10}{13} \cdot \frac{12}{51} + \frac{3}{13} \cdot \frac{11}{51} \\ &= \frac{120}{663} + \frac{33}{663} \\ &= \frac{153}{663} \\ &= \frac{51}{221} \\ &\approx 0.2308 \text{ or } 23.08\% \end{align*}\]

This means that given we’ve already drawn a heart, there’s approximately a 23.08% chance of drawing a face card on the next draw.

c. Combined Probability of Drawing a Heart and Then a Face Card: To find this joint probability, we multiply the probability of drawing a heart by the conditional probability of drawing a face card given that we drew a heart:

\[\begin{align*} P(A \text{ and } B) &= P(A) \times P(B|A) \\ &= \frac{13}{52} \times \frac{51}{221} \\ &= \frac{663}{11492} \\ &\approx 0.0577 \text{ or } 5.77\% \end{align*}\]

Note

This problem demonstrates the importance of considering all possible scenarios in conditional probability. The probability of drawing a face card on the second draw depends on whether the first heart drawn was a face card or not. By accounting for both possibilities, we arrive at a more accurate calculation.