Bayes' Theorem
Event-probability form
Bayes' theorem tells you how to update the probability of a hypothesis after observing evidence. The posterior, , combines the prior, , with the likelihood, , and divides by the evidence probability, . In plain language: start with how plausible the hypothesis was, account for how well it predicts the observation, and normalize the result.
Notes
Here is a hypothesis represented as an event, and is the observed-data event. The conditioning bar means "given," not division. The likelihood is the probability of the observed data assuming the hypothesis, not the probability that the hypothesis is true.
The denominator accounts for how probable the observation is overall; for an exhaustive, mutually exclusive collection of hypotheses , it can be calculated as . This event-probability form requires and an appropriately defined likelihood. Continuous-data versions use probability densities rather than probabilities of exact observations.
Related equations
- Shannon Entropyrelated
Sources
- Probabilistic Modeling and Bayesian AnalysisMIT OCW lecture notes.