Learn & Review: Linear Algebra for Beginners to Experts

Linear Algebra: Lecture 1 - Introduction and Fundamental Problem

This lecture introduces MIT's 18.06 Linear Algebra course, focusing on the fundamental problem of solving systems of linear equations. It outlines three perspectives for understanding this problem: the row picture, the column picture, and the matrix form.

1. The Fundamental Problem: Solving Linear Equations

The core task is to solve a system of linear equations, typically represented as $Ax = b$, where $A$ is the matrix of coefficients, $x$ is the vector of unknowns, and $b$ is the vector of right-hand sides. The lecture starts with the case of $n$ equations and $n$ unknowns.

2. The Row Picture

• Concept: Visualizing one equation at a time.
• For 2x2 systems: Each linear equation represents a line in the $xy$-plane. The solution to the system is the point where these lines intersect.
• Example: For the system: $2x - y = 0$ $-x + 2y = 3$ The first equation represents a line passing through the origin (0,0) and (1,2). The second equation represents a line passing through (-3,0) and (-1,1). The intersection point (1,2) is the solution.
• For 3x3 systems: Each linear equation represents a plane in 3D space ($xyz$). The solution to the system is the point where these three planes intersect.
• Limitation: As the number of equations and dimensions increase, visualizing the intersection of planes becomes difficult.

3. The Column Picture

• Concept: Visualizing the problem as a linear combination of the columns of the matrix $A$.
• Representation: The equation $Ax = b$ can be rewritten as a sum of the columns of $A$, each multiplied by the corresponding component of the vector $x$. $x_1 \cdot (\text{column } 1) + x_2 \cdot (\text{column } 2) + \dots + x_n \cdot (\text{column } n) = b$
• For 2x2 systems: The problem is to find a linear combination of the two column vectors of matrix $A$ that equals the vector $b$.
  • Example: For the matrix $A = \begin{pmatrix} 2 & -1 \ -1 & 2 \end{pmatrix}$ and $b = \begin{pmatrix} 0 \ 3 \end{pmatrix}$, the equation is $x \begin{pmatrix} 2 \ -1 \end{pmatrix} + y \begin{pmatrix} -1 \ 2 \end{pmatrix} = \begin{pmatrix} 0 \ 3 \end{pmatrix}$.
  • The lecture visually demonstrates combining the column vectors (2, -1) and (-1, 2) to produce the target vector (0, 3). The solution $x=1, y=2$ means taking one of the first column and two of the second column.
• For 3x3 systems: The problem is to find a linear combination of three 3D column vectors that equals the vector $b$.
  • Example: For the matrix $A = \begin{pmatrix} 2 & -1 & 0 \ -1 & 2 & -1 \ 0 & -3 & 4 \end{pmatrix}$ and $b = \begin{pmatrix} 0 \ -1 \ 4 \end{pmatrix}$, the equation is $x \begin{pmatrix} 2 \ -1 \ 0 \end{pmatrix} + y \begin{pmatrix} -1 \ 2 \ -3 \end{pmatrix} + z \begin{pmatrix} 0 \ -1 \ 4 \end{pmatrix} = \begin{pmatrix} 0 \ -1 \ 4 \end{pmatrix}$.
  • In this specific example, the target vector $b$ is identical to the third column of $A$. Thus, the solution is $x=0, y=0, z=1$.
• Key Question: Do the linear combinations of the columns fill the entire space? This relates to whether a solution exists for every possible right-hand side vector $b$.
  • If the columns are linearly independent (e.g., they don't lie in the same plane in 3D), their combinations can fill the entire space, meaning a solution exists for every $b$. Such matrices are called non-singular or invertible.
  • If the columns are linearly dependent (e.g., one column is a combination of others, or all columns lie in the same plane), their combinations will only fill a subspace, and solutions will not exist for all $b$. Such matrices are singular.

4. The Matrix Form ($Ax = b$)

• Definition: A matrix is a rectangular array of numbers.
• Matrix-Vector Multiplication: There are two ways to view $Ax$:
  1. Linear Combination of Columns: $Ax$ is a linear combination of the columns of $A$, with coefficients from the vector $x$. This is the preferred view for understanding the column picture.
     • Example: $\begin{pmatrix} 2 & 5 \ 1 & 3 \end{pmatrix} \begin{pmatrix} 1 \ 2 \end{pmatrix} = 1 \begin{pmatrix} 2 \ 1 \end{pmatrix} + 2 \begin{pmatrix} 5 \ 3 \end{pmatrix} = \begin{pmatrix} 2 \ 1 \end{pmatrix} + \begin{pmatrix} 10 \ 6 \end{pmatrix} = \begin{pmatrix} 12 \ 7 \end{pmatrix}$.
  2. Dot Products of Rows: $Ax$ can be computed by taking the dot product of each row of $A$ with the vector $x$.
     • Example: For the same matrix and vector:
       • Row 1: $(2 \cdot 1) + (5 \cdot 2) = 2 + 10 = 12$
       • Row 2: $(1 \cdot 1) + (3 \cdot 2) = 1 + 6 = 7$ Result: $\begin{pmatrix} 12 \ 7 \end{pmatrix}$.

5. Looking Ahead

• The next lecture will cover elimination, a systematic method for finding solutions to $Ax = b$ and determining when solutions exist.
• The course will explore the properties of matrices, particularly whether they are singular or non-singular, which determines if $Ax = b$ has a solution for every $b$.
• The concept of linear combinations of vectors is fundamental and will be revisited throughout the course, even in higher dimensions where visualization is challenging.