Mathematical Statistics

Units

1. 1

   Sampling distributions

   4 lessons

   1. Populations, samples and statisticsDistinguish a population parameter from a sample statistic and find the sampling distribution of a two-draw mean.
   2. Sampling distribution of the meanFind the center and standard error of a sample mean, and decide whether a normal model for it is exact or approximate.
   3. Chi-square, t and F distributionsConstruct χ², t and F statistics from independent normal samples and identify their reference distributions and degrees of freedom.
   4. Order statisticsFind cutoff probabilities for the minimum, maximum, k-th ordered value, and odd-sample median of independent, identically distributed draws.
2. 2

   Point estimation

   7 lessons

   1. Estimators and estimatesTell an estimator (a statistic, a random variable) from an estimate (its value on one sample), and use X̄, S² and the sample proportion p̂ to estimate μ, σ² and p.
   2. Bias, variance and MSECompute the bias and variance of an estimator, combine them as MSE = Var + bias², show that S² is unbiased for σ² because it divides by n − 1, and see that an unbiased estimator need not have the smallest MSE.
   3. Consistency, efficiency and sufficiencyShow an estimator is consistent, compare two estimators by relative efficiency, compute the Cramér-Rao lower bound from the Fisher information, and find a sufficient statistic by the factorization theorem.
   4. The method of momentsBuild moment estimators by equating sample moments to population moments and solving for the parameters, for the exponential rate and for the normal mean and variance, and judge one by simulation.
   5. The likelihood functionWrite the likelihood L(θ) as the joint density or pmf of the data read as a function of θ, plot it for Bernoulli data, compare parameter values by likelihood ratios, and turn the product into a sum with the log-likelihood.
   6. Finding maximum likelihood estimatorsFind maximum likelihood estimators by setting the score to zero and checking the second derivative, for Bernoulli, Poisson, exponential and normal models, and find the MLE at the boundary for uniform(0, θ).
   7. Properties of the MLEUse the invariance of the MLE, state its consistency and asymptotic normality with variance 1/(nI(θ)), and compute a standard error from the observed information.
3. 3

   Sufficiency and efficiency

   3 lessons

   1. Sufficient statisticsA statistic is sufficient when the data's conditional distribution given it does not depend on θ; the factorization theorem finds Σxᵢ for Bernoulli and Poisson samples and (Σxᵢ, Σxᵢ²) for the normal.
   2. Fisher information and the Cramér-Rao boundFisher information I(θ) as the curvature of the log-likelihood, the lower bound 1/(nI(θ)) on the variance of unbiased estimators, and the sample proportion that reaches it, with sharp and flat likelihoods compared.
   3. Rao-Blackwell and best unbiased estimatorsConditioning an unbiased estimator on a sufficient statistic never raises its variance; with completeness this gives the minimum variance unbiased estimator, worked for e^{−λ} from a Poisson sample.
4. 4

   Confidence intervals

   3 lessons

   1. Confidence intervals and pivotsA 95% confidence interval as a procedure covering theta in 95% of samples, built from a pivotal quantity, shown with many simulated intervals.
   2. Intervals for a meanThe z interval with known sigma and the t interval X-bar +/- t* S/sqrt(n) for unknown sigma, and choosing sample size from a target margin of error.
   3. Intervals for proportions and variancesWald and Wilson intervals for a proportion, the chi-square interval for a normal variance, and large-sample intervals from asymptotic normality of the MLE.
5. 5

   Hypothesis testing

   6 lessons

   1. The logic of hypothesis testsNull and alternative hypotheses, test statistic, rejection region, type I and type II errors in a 2 by 2 table, and the significance level alpha.
   2. P-valuesThe p-value as the probability under H0 of a result at least as extreme, one-sided versus two-sided tests, common misreadings, and duality with confidence intervals.
   3. Power and sample sizeThe power function, the probability of rejecting H0 as a function of theta, plotted for a one-sided z test, and choosing n to reach a target power.
   4. The Neyman-Pearson lemmaThe most powerful test of a simple null against a simple alternative rejects when the likelihood ratio is large, proved and applied to a normal mean.
   5. Generalized likelihood ratio testsLambda as the maximized likelihood under H0 over the unrestricted maximum, rejecting for small Lambda, derived for a normal mean with unknown variance to recover the t test.
   6. One and two sample t testsOne-sample, paired and two-sample t tests (pooled and Welch), their assumptions, and a worked comparison of two group means.
6. 6

   Linear models

   4 lessons

   1. The simple linear regression modelWrite and interpret a simple linear regression model, find its conditional mean at a given study time, and calculate a score’s signed departure from that mean.
   2. Least squares estimatesCalculate a least-squares slope and intercept from a small data set and explain why they minimize squared vertical residuals.
   3. Sampling properties of the estimatorsShow that b₁ and b₀ are unbiased under the regression model, derive Var(b₁) = σ²/Sxx, and see how the spread of x controls the precision of the slope.
   4. Tests and intervals for the slopeTest H₀: β₁ = 0 with t = b₁/SE(b₁) on n − 2 degrees of freedom, build a confidence interval for β₁, and read both from a regression output table.
7. 7

   Analysis of variance

   3 lessons

   1. One-way analysis of varianceSplits total variation into between-group and within-group sums of squares for three fertilizers' yields, forms F = MSB/MSW with (k − 1, N − k) degrees of freedom, and reads the p-value from the F distribution.
   2. Multiple comparisonsWhy many pairwise t-tests inflate the familywise error rate, the Bonferroni correction, and Tukey's intervals for every pair of means, drawn as intervals on one axis.
   3. Two-way analysis of varianceTwo factors with interaction, the interaction plot whose crossing lines signal it, the decomposition into main effects, interaction and error, and the three F tests.
8. 8

   Goodness of fit and contingency tables

   2 lessons

   1. Chi-square goodness of fitPearson's statistic Σ(O − E)²/E for a die and for a Poisson model of goal counts, with k − 1 − m degrees of freedom when m parameters are estimated, observed and expected drawn as paired bars.
   2. Contingency tablesTests of independence and of homogeneity in an r × c table, expected counts from the margins, (r − 1)(c − 1) degrees of freedom, and the likelihood-ratio G statistic beside Pearson's.
9. 9

   Nonparametric methods and the bootstrap

   4 lessons

   1. Rank-based testsThe sign test, Wilcoxon signed-rank test and Mann-Whitney (Wilcoxon rank-sum) test as distribution-free alternatives to t tests, with a worked ranking table.
   2. Permutation testsExact and Monte Carlo permutation tests for a difference in means, the null permutation distribution as a histogram, and the exchangeability assumption.
   3. Empirical CDF and Kolmogorov-SmirnovThe empirical CDF, the Glivenko-Cantelli theorem stated, and the Kolmogorov-Smirnov statistic as the largest gap between the empirical and hypothesized CDFs.
   4. The bootstrapEfron's nonparametric bootstrap: resampling with replacement to estimate standard errors, and percentile bootstrap confidence intervals, illustrated for a sample median.