Conditional Probability & Bayes' Theorem

Conditional probability

$P(A \mid B)$ is the probability of $A$ given that $B$ has happened:

$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$

The symbol $\cap$ means "and" (both happen). Rearranging gives the multiplication rule:

$P(A \cap B) = P(A \mid B)\,P(B)$

Independence, revisited

$A$ and $B$ are independent when knowing one tells you nothing about the other: $P(A \mid B) = P(A)$. In that case the multiplication rule simplifies to $P(A \cap B) = P(A)\,P(B)$.

Two-way tables make it concrete

Suppose 100 people are classified by whether they exercise and whether they sleep well:

	Sleeps well	Poor sleep	Total
Exercises	40	10	50
No exercise	20	30	50
Total	60	40	100

• $P(\text{sleeps well}) = \frac{60}{100} = 0.6$
• $P(\text{sleeps well} \mid \text{exercises}) = \frac{40}{50} = 0.8$

Since $0.8 \neq 0.6$, sleeping well and exercising are not independent — exercisers sleep well more often.

Law of total probability

If $B$ can occur through disjoint cases $A$ and "not $A$" (written $A^c$):

$P(B) = P(B \mid A)\,P(A) + P(B \mid A^c)\,P(A^c)$

Bayes' theorem

Bayes flips a conditional probability around:

$P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}$

where $P(B)$ comes from the law of total probability. It's how you go from "probability of a positive test given disease" to the thing you actually want: "probability of disease given a positive test."

The medical-test surprise

A disease affects $1\%$ of people. A test is $99\%$ accurate on the sick (true positive) and has a $5\%$ false-positive rate on the healthy. You test positive. What's the chance you actually have it?

Let $D$ = disease and $+$ = positive test.

• $P(D) = 0.01$, so $P(D^c) = 0.99$
• $P(+ \mid D) = 0.99$, $P(+ \mid D^c) = 0.05$

First the total probability of a positive test:

$P(+) = 0.99(0.01) + 0.05(0.99) = 0.0099 + 0.0495 = 0.0594$

Now Bayes:

$P(D \mid +) = \frac{0.99 \cdot 0.01}{0.0594} \approx 0.167$

Only about 17%. Because the disease is rare, the few real cases are swamped by false positives from the huge healthy group. Ignoring this is the base-rate fallacy.

Key takeaways

• $P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}$; rearrange it for the multiplication rule.
• Independent means $P(A \mid B) = P(A)$.
• Two-way tables turn conditional probabilities into simple ratios.
• Bayes' theorem flips a conditional using the base rate — and rare conditions make even accurate tests produce mostly false positives.