Linear Algebra

Units

1. 1

   Linear systems and row reduction

   3 lessons

   1. Row and column picturesDraw the system Ax = b two ways, as lines (or planes) that meet in the solution and as columns of A combined to reach b, and tell from either picture whether there are zero, one or infinitely many solutions.
   2. Gaussian eliminationReduce a system Ax = b to upper triangular form with row operations on the augmented matrix, read off the pivots, and finish by back substitution, including what happens when a pivot is zero.
   3. Reduced row echelon formReduce an augmented matrix to reduced row echelon form, name the pivot and free variables, and write the complete solution as a particular solution plus combinations of null space vectors.
2. 2

   Span and solution sets

   3 lessons

   1. Vector equations and spanRewrite a linear system as a vector equation x₁a₁ + x₂a₂ = b, describe the span of one or two vectors in ℝ² and ℝ³ as a line or a plane through the origin, and decide by row reduction whether b lies in the span.
   2. The matrix equation Ax = bRead Ax = b as a combination of the columns of A with weights x, and show that Ax = b has a solution for every b exactly when A has a pivot in every row, so that the columns span ℝᵐ.
   3. Solution sets in parametric vector formWrite the solutions of Ax = 0 as the span of vectors through the origin, write the solutions of Ax = b as one particular solution plus that span, and draw both as parallel lines or planes.
3. 3

   Independence and linear transformations

   3 lessons

   1. Linear independenceDecide whether a list of vectors is independent from the definition and by elimination, and recognise a dependent column in a data matrix.
   2. Linear transformationsTest whether a map is linear, build its matrix from the images of the basis vectors, and picture it as a transformation of the unit square and of a grid.
   3. Rotations, reflections and projectionsWrite the standard 2 × 2 matrices for rotation, reflection, scaling, shear and projection by reading the images of e₁ and e₂, and describe each by its effect on the unit square.
4. 4

   Matrix algebra

   5 lessons

   1. Multiplication as compositionRead AB as "apply B, then A", compute the product with the row-column rule after checking the sizes fit, and show with a shear and a rotation that AB and BA differ.
   2. Transpose and special matricesForm the transpose, use (AB)ᵀ = BᵀAᵀ, recognise symmetric, diagonal, identity and triangular matrices, and read the Gram matrix XᵀX of a data matrix as the table of dot products of its columns.
   3. The inverse matrixDefine A⁻¹ by A A⁻¹ = I, find it for a 2 × 2 matrix by the formula and by Gauss-Jordan elimination on [A | I], and use (AB)⁻¹ = B⁻¹A⁻¹.
   4. When a matrix is invertibleDecide whether a square matrix is invertible from its determinant, its rank, its null space or its columns, and see why these tests all say the same thing.
   5. LU factorizationRecord the multipliers of elimination in a lower triangular matrix L so that A = LU with U the upper triangular result, and solve Ax = b by forward then back substitution, cheaply for many right sides.
5. 5

   Determinants

   3 lessons

   1. Determinants by cofactor expansionCompute 2 × 2 and 3 × 3 determinants, expand a determinant along any row or column with the cofactor signs, and use zeros and triangular form to shorten the work.
   2. Properties of determinantsTrack how each row operation changes a determinant, compute determinants by row reduction, and use det(AB) = det A det B, det Aᵀ = det A and the test det A ≠ 0 for invertibility.
   3. Determinants as area and volumeRead |det A| as the area of the parallelogram (or volume of the parallelepiped) on the columns, use it as the factor by which x ↦ Ax scales every area, read orientation from the sign, and solve a 2 × 2 system by Cramer's rule.
6. 6

   Subspaces, bases and dimension

   5 lessons

   1. Vector spacesState the vector space rules, recognise polynomials and functions as vectors, and test whether a subset of a space is a subspace.
   2. Column space and null spaceDescribe the column space C(A) and the null space N(A) of a matrix, and use them to decide whether Ax = b is solvable and whether its solution is unique.
   3. Basis and dimensionDefine a basis as an independent spanning set, define dimension as the number of vectors in a basis, and read bases for the column space and null space off the reduced row echelon form.
   4. The rank-nullity theoremState and use rank(A) + dim N(A) = n, read as pivots plus free columns, and apply it to solution sets and to systems with more unknowns than equations.
   5. Coordinates and change of basisWrite a vector in coordinates relative to a basis, convert between standard and skewed coordinates with the change of basis matrix, and see why a well-chosen basis makes coordinates easy.
7. 7

   Eigenvalues and eigenvectors

   3 lessons

   1. Eigenvalues and eigenvectorsDefine an eigenvector and eigenvalue by Av = λv, find them for simple 2 × 2 matrices by inspection and checking, and see which matrices have real eigen-directions at all.
   2. The characteristic polynomialFind all eigenvalues of a matrix from det(A − λI) = 0, find eigenvectors as the null space of A − λI, and use trace and determinant as the sum and product of the eigenvalues.
   3. Diagonalizing a matrixWrite a matrix with n independent eigenvectors as A = SΛS⁻¹, use it to compute powers Aᵏ = SΛᵏS⁻¹, and find the steady state of a Markov chain from the eigenvector for λ = 1.
8. 8

   Complex eigenvalues and dynamics

   3 lessons

   1. Complex eigenvaluesFind complex eigenvalues λ = a ± bi of a real 2 × 2 matrix, write a rotation-scaling matrix as r times a rotation by φ, factor A = PCP⁻¹, and predict the spiral of repeated images from |λ|.
   2. Discrete dynamical systemsIterate xₖ₊₁ = Axₖ, write xₖ in the eigenvector basis as c₁λ₁ᵏv₁ + c₂λ₂ᵏv₂, and read the long-run behaviour from the eigenvalues: attractor, repeller or saddle.
   3. Markov chains and steady statesMove a probability vector one step with a stochastic matrix, find the steady state as the eigenvector for λ = 1 with entries summing to 1, and explain why a regular chain approaches it from any start.
9. 9

   Orthogonality and least squares

   6 lessons

   1. The dot productCompute dot products, angles and cosine similarity for nonzero vectors; recognize orthogonal vectors.
   2. Orthogonal vectors and subspacesTest orthogonality with the dot product, find the orthogonal complement of a subspace, see that the row space is perpendicular to the null space, and use orthonormal bases to read coordinates as dot products.
   3. Projection onto a subspaceProject b onto the column space of A by solving AᵀA x̂ = Aᵀb, build P = A(AᵀA)⁻¹Aᵀ, check P² = P and Pᵀ = P, and split b = p + e with e orthogonal to the subspace.
   4. Gram-Schmidt and QRTurn independent columns into orthonormal ones with Gram-Schmidt, record the steps as A = QR with R upper triangular, and use orthogonal matrices (QᵀQ = I) that preserve lengths.
   5. The normal equationsWhen Ax = b has no solution, find the x̂ that minimizes |Ax − b|² by solving AᵀA x̂ = Aᵀb, and read the error e = b − Ax̂ as orthogonal to every column of A.
   6. Fitting lines and curvesBuild the design matrix A for a line and for a parabola, solve AᵀA x̂ = Aᵀb, and read the fit and its residuals; see why a model with as many columns as points fits exactly yet predicts badly.
10. 10

    Symmetric matrices

    2 lessons

    1. Symmetric matrices and the spectral theoremShow that a real symmetric matrix has real eigenvalues and perpendicular eigenvectors, write it as A = QΛQᵀ with Q orthogonal, and expand it as a sum λ₁q₁q₁ᵀ + λ₂q₂q₂ᵀ of projections.
    2. Positive definite matricesDecide whether a symmetric matrix is positive definite by its eigenvalues, its pivots or xᵀAx > 0, draw its quadratic form as nested ellipses, and see why Gram and covariance matrices are positive semidefinite.
11. 11

    Orthogonal matrices and quadratic forms

    4 lessons

    1. Orthogonal matricesRecognize an orthogonal matrix by QᵀQ = I, use Q⁻¹ = Qᵀ, show that Q preserves lengths and angles, and identify every 2 × 2 orthogonal matrix as a rotation or a reflection.
    2. Quadratic formsWrite a quadratic form as Q(x) = xᵀAx with A symmetric, remove its cross term by the change of variable x = Py with P orthogonal, and classify it as positive definite, negative definite or indefinite from the eigenvalues of A, with its level curves.
    3. Principal axes of a conicRotate the conic 5x₁² − 4x₁x₂ + 5x₂² = 48 to its principal axes, read its semi-axes from the eigenvalues of A, and tell an ellipse from a hyperbola by their signs.
    4. Constrained optimization of quadratic formsFind the largest and smallest values of xᵀAx on the unit circle and say where they occur: they are the extreme eigenvalues of A, at their unit eigenvectors.