Probability

Units

1. 1

   Sample spaces and the axioms

   4 lessons

   1. Sample spaces and eventsDescribe an experiment by its sample space Ω, treat events as subsets of Ω, and count unions, intersections and complements on the two-dice grid, with De Morgan's laws.
   2. The axioms of probabilityState Kolmogorov's three axioms for P, derive P(Aᶜ) = 1 − P(A), P(∅) = 0 and monotonicity from them, and prove P(A ∪ B) = P(A) + P(B) − P(A ∩ B) on the two-dice grid.
   3. Equally likely outcomesUse the naive definition P(A) = |A| / |Ω| for a finite sample space of equally likely outcomes, and say when it applies and when it gives a wrong answer.
   4. Probability as long-run frequencyCalculate a running relative frequency and distinguish it from the probability of heads on the next toss.
2. 2

   Counting methods

   4 lessons

   1. The multiplication rule and permutationsCount outcomes of sequential choices with the multiplication rule, count orderings with n! and ordered samples with n!/(n − k)!, and use them for the birthday problem.
   2. Combinations and binomial coefficientsCount unordered selections with C(n, k) = n!/(k!(n − k)!), read Pascal's triangle and prove identities by counting, and compute poker hand probabilities.
   3. Sampling with and without replacementCount samples of size k from n objects in all four cases (ordered or not, with or without replacement), including stars and bars C(n + k − 1, k), and know which case gives equally likely outcomes.
   4. Inclusion-exclusion and matchingCompute the probability of a union of n events by inclusion-exclusion and solve de Montmort's matching problem, whose answer tends to 1 − 1/e.
3. 3

   Conditional probability and Bayes' rule

   4 lessons

   1. Conditional probabilityDefine P(A|B) = P(A and B)/P(B) as probability on the sample space restricted to B, use the multiplication rule P(A and B) = P(B)P(A|B), and keep P(A|B) apart from P(B|A).
   2. The law of total probabilityPartition the sample space, compute P(A) = ∑ P(A|Bᵢ)P(Bᵢ) as a weighted average, and use it on a multistage urn problem.
   3. Bayes' ruleReverse a conditional probability with P(A|B) = P(B|A)P(A)/P(B), using prevalence, sensitivity and specificity, and explain why a positive test for a rare disease still leaves a low probability of disease.
   4. Conditioning paradoxesSolve the Monty Hall problem by conditioning, spot the prosecutor's fallacy of confusing P(A|B) with P(B|A), and explain Simpson's paradox with a two-group admissions table.
4. 4

   Independence

   3 lessons

   1. Independent eventsDefine independence as P(A and B) = P(A)P(B), distinguish pairwise from mutual independence with a counterexample, and separate independent events from disjoint events.
   2. Conditional independenceDefine independence given a third event C, show that events can be independent given C yet dependent overall, and that events independent overall can become dependent once C is known.
   3. Independent trials and gambler's ruinMultiply probabilities along independent trials, use P(at least one success) = 1 − (1 − p)ⁿ, and solve gambler's ruin by first-step analysis.
5. 5

   Discrete random variables

   5 lessons

   1. Random variables and PMFsTreat a random variable as a function from outcomes to numbers, write its probability mass function and its CDF F(x) = P(X ≤ x), and read probabilities of events from both.
   2. Bernoulli and binomial distributionsDefine Bernoulli(p) and Binomial(n, p), derive the binomial PMF (n choose k)pᵏ(1 − p)ⁿ⁻ᵏ from independent trials, and read how n and p shape its bar chart, mean np and variance np(1 − p).
   3. Geometric and negative binomial distributionsDefine the geometric distribution for the trial of the first success, derive its PMF (1 − p)ᵏ⁻¹p and the memoryless property, and extend to the negative binomial for the trial of the r-th success.
   4. The Poisson distributionDerive the Poisson PMF e⁻λ λᵏ/k! as the limit of Binomial(n, λ/n), use it for counts of rare events, and use its mean and variance, both λ.
   5. The hypergeometric distributionDerive the hypergeometric PMF for the number of successes when sampling without replacement, use it for capture-recapture, and see when the binomial is a good approximation.
6. 6

   Expectation and variance

   3 lessons

   1. Expected valueDefine E(X) = ∑ x P(X = x) as a probability-weighted average, prove linearity of expectation even for dependent variables, and use indicator variables to count expected matches.
   2. Variance and LOTUSUse the law of the unconscious statistician E(g(X)) = ∑ g(x)p(x), define variance and standard deviation, derive Var(X) = E(X²) − (E(X))² and Var(aX + b) = a²Var(X), and apply them to the binomial and Poisson.
   3. Markov and Chebyshev inequalitiesState and prove Markov's inequality P(X ≥ a) ≤ E(X)/a and Chebyshev's inequality P(|X − μ| ≥ kσ) ≤ 1/k², and compare both bounds with exact binomial tail probabilities.
7. 7

   Continuous random variables

   4 lessons

   1. Probability density functionsTreat a continuous random variable through its density f, compute P(a < X < b) as an area, build the CDF F(x) = ∫ f, and find E(X) and Var(X) by integration.
   2. Uniform and exponential distributionsUse the Uniform(a, b) and Exponential(λ) densities, find their CDFs, means and variances, prove the exponential's memoryless property and connect it to waiting times in a Poisson process.
   3. The normal distributionUse the Normal(μ, σ²) density, standardize with Z = (X − μ)/σ, find probabilities and percentiles with the standard normal CDF Φ, and apply the 68-95-99.7 rule.
   4. Functions of a random variableFind the distribution of Y = g(X) by the CDF method and by the change of variables formula f_Y(y) = f_X(x)|dx/dy|, and use it for the log-normal and for the universality of the uniform.
8. 8

   Joint distributions

   4 lessons

   1. Joint and marginal distributionsDescribe two random variables together by a joint PMF or joint density, recover each variable's marginal distribution by summing or integrating out the other, and compute probabilities of events about both.
   2. Conditional distributions and independenceDefine conditional PMFs and densities f(y | x) = f(x, y)/f_X(x), test independence of random variables by factorization of the joint distribution, and compute expectations of g(X, Y) by two-dimensional LOTUS.
   3. Conditional expectationDefine E(Y | X = x) and the random variable E(Y | X), state and use Adam's law E(Y) = E(E(Y | X)) and Eve's law Var(Y) = E(Var(Y | X)) + Var(E(Y | X)), and apply them to a random sum.
   4. Sums of independent random variablesFind the distribution of X + Y for independent X and Y by convolution, and use it to show that sums of independent Poissons are Poisson, sums of independent normals are normal, and the sum of two uniforms has a triangular density.
9. 9

   Covariance and correlation

   3 lessons

   1. CovarianceCompute Cov(X, Y) = E(XY) − E(X)E(Y), use its bilinearity, and find Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y), applied to the variance of a hypergeometric count.
   2. CorrelationCompute Corr(X, Y) = Cov(X, Y)/(σ_X σ_Y), prove that it lies between −1 and 1 with equality only for a linear relation, read it from scatter plots, and explain why zero correlation does not imply independence.
   3. Multinomial and bivariate normalCompute multinomial probabilities for category counts, derive Cov(Nᵢ, Nⱼ) = −npᵢpⱼ, and read the bivariate normal density through its elliptical contours, whose tilt and shape are set by ρ.
10. 10

    Limit theorems

    3 lessons

    1. The law of large numbersWeak law of large numbers proved with Chebyshev's inequality, the strong law stated, and a simulation of sample means settling at the true mean.
    2. The central limit theoremThe CLT: the standardized sample mean converges in distribution to N(0,1), shown with histograms of means from a skewed distribution as n grows.
    3. Normal approximation to the binomialApproximating Binomial(n, p) by Normal(np, np(1-p)) with continuity correction, rules of thumb for when it works, and a worked polling example.
11. 11

    Moment generating functions

    3 lessons

    1. Moment generating functionsDefinition M(t) = E(e^(tX)), moments from derivatives at zero, and the MGFs of the Bernoulli, binomial, Poisson, exponential and normal distributions.
    2. MGFs of sums and uniquenessThe MGF of a sum of independent variables is the product of their MGFs; the uniqueness theorem; and using both to show sums of independent normals are normal.
    3. Proving the CLT with MGFsSketch of the central limit theorem proof using the MGF of the standardized mean, a Taylor expansion, and the limit (1 + t^2/(2n))^n tending to e^(t^2/2).
12. 12

    Random vectors and the multivariate normal

    2 lessons

    1. Random vectors and covariance matricesThe mean vector and covariance matrix of a random vector, Cov(AX + b) = A Σ Aᵀ, and the covariance matrix as a symmetric positive semidefinite matrix whose eigenvectors set the scatter's axes.
    2. The multivariate normal distributionThe density of a bivariate normal drawn as elliptical level curves, marginals and conditionals that are normal, and uncorrelated jointly normal variables that are independent.
13. 13

    Random walks, Poisson processes and Markov chains

    3 lessons

    1. Random walksThe simple random walk as a sum of ±1 steps, its mean and variance, sample paths spreading like √n, and the gambler's ruin probability by first-step analysis.
    2. The Poisson processArrivals at rate λ: counts in disjoint intervals are independent Poisson, gaps between arrivals are exponential, and the memoryless property, drawn as arrival times on a timeline.
    3. Markov chainsA transition matrix and its diagram, n-step probabilities as matrix powers, and the stationary distribution πP = π reached by a weather chain from any start.