Probability, from coin flips to Bayes.

Probability is a number between 0 and 1 that measures how likely an event is to occur. A probability of 0 means impossible. A probability of 1 means certain. Everything real lives somewhere between.

We write the probability of event A as P(A). For a fair coin, P(Heads) = 0.5. For rolling a 6 on a fair die, P(6) = 1/6 ≈ 0.167.

There are three main interpretations of probability: classical (count equally likely outcomes), frequentist (long-run proportion), and Bayesian (degree of belief). All three are useful. We'll touch all of them.

The sample space Ω (omega) is the set of all possible outcomes of an experiment. An event is any subset of the sample space — a collection of outcomes we care about.

For a die: Ω = {1, 2, 3, 4, 5, 6}. The event "roll an odd number" = {1, 3, 5}. Its probability is 3/6 = 1/2.

Events can be combined with set operations: the union A ∪ B ("A or B"), the intersection A ∩ B ("A and B"), and the complement Aᶜ ("not A").

The probability that event A or event B occurs is given by the addition rule:

We subtract the intersection because outcomes in both A and B would otherwise be counted twice. If A and B are mutually exclusive (they can't both happen), then P(A ∩ B) = 0 and the rule simplifies to P(A ∪ B) = P(A) + P(B).

Two events are independent when knowing one occurred tells you nothing about the other. For independent events: P(A ∩ B) = P(A) · P(B).

Conditional probability answers: "given that B happened, how likely is A?" We write this P(A | B), read "probability of A given B."

This makes sense geometrically: we've restricted our universe to the outcomes inside B. Now we ask what fraction of those outcomes are also in A.

From the definition we get the multiplication rule: P(A ∩ B) = P(A | B) · P(B). This is one of the most-used identities in all of probability.

Bayes' theorem lets us reverse conditional probabilities. We know P(B | A) — perhaps it's easy to measure — but we want P(A | B), which is often the more useful quantity.

P(A) is the prior — what we believed before seeing B. P(A | B) is the posterior — our updated belief after seeing B. Bayes' theorem is the engine of rational belief update.

A probability distribution describes how probability is spread across all possible outcomes. For discrete outcomes (like die faces) we use a probability mass function (PMF). For continuous outcomes (like heights) we use a probability density function (PDF).

Two of the most important discrete distributions:

Binomial(n, p) — the number of successes in n independent trials, each with success probability p. Rolling a 6 exactly 3 times in 10 dice rolls follows a binomial distribution.

Expected value E[X] = Σ x · P(X = x) — the long-run average outcome. For a fair die: E[X] = (1+2+3+4+5+6)/6 = 3.5.

The Law of Large Numbers (LLN) says that as the number of trials grows, the sample mean converges to the true expected value. The more you repeat an experiment, the less luck matters.

This is not the gambler's fallacy. The LLN does not say that after a long run of heads, tails is "due." Each flip is independent. The law operates through dilution — later flips don't correct earlier ones; they simply outnumber them.