Lec 04-28-2026: Projections & Lines and Planes

Recall

In our last lecture, we applied the Law of Cosines to vector subtraction in order to motivate the notion of the dot product.

If $\vec{u}, \vec{v} \in \mathbb{R}^n$ where $\vec{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix}$ & $\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix}$, then

$$\vec{u}^T\vec{v} = \begin{bmatrix} u_1 & u_2 & \cdots & u_n \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} := u_1v_1 + u_2v_2 + \cdots + u_nv_n$$

Additionally, from two lectures ago, we said:

If $\vec{w} = \begin{bmatrix} w_1 \\ w_2 \\ \vdots \\ w_n \end{bmatrix} \in \mathbb{R}^n$, then $\|\vec{w}\| = \sqrt{w_1^2 + w_2^2 + \cdots + w_n^2}$, so $\|\vec{w}\|^2 = w_1^2 + w_2^2 + \cdots + w_n^2$.

Notice:

$$\vec{w}^T\vec{w} = \begin{bmatrix} w_1 & w_2 & \cdots & w_n \end{bmatrix} \begin{bmatrix} w_1 \\ w_2 \\ \vdots \\ w_n \end{bmatrix} = w_1^2 + w_2^2 + \cdots + w_n^2$$

Hence $\|\vec{w}\|^2 = \vec{w}^T\vec{w}$, or equivalently $\|\vec{w}\| = \sqrt{\vec{w}^T\vec{w}}$.

Properties of the Dot Product

• $\vec{u}^T(\vec{v} + \vec{w}) = \vec{u}^T\vec{v} + \vec{u}^T\vec{w}$
• $\vec{0}^T\vec{w} = 0$ (a real number, not a vector)
• $\vec{w}^T\vec{v} = \vec{v}^T\vec{w}$
• $(c\vec{u})^T\vec{v} = c(\vec{u}^T\vec{v}) = \vec{u}^T(c\vec{v})$ (scaling $\vec{u}$ by some factor $c$)
• $(c\vec{w})^T = c\vec{w}^T$

Justification of Property 1

LHS $= \vec{u}^T(\vec{v} + \vec{w})$:

$$\begin{align*} \vec{u}^T(\vec{v} + \vec{w}) &= \begin{bmatrix} u_1 & u_2 & \cdots & u_n \end{bmatrix} \left( \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} + \begin{bmatrix} w_1 \\ w_2 \\ \vdots \\ w_n \end{bmatrix} \right) \\ &= \begin{bmatrix} u_1 & u_2 & \cdots & u_n \end{bmatrix} \begin{bmatrix} v_1 + w_1 \\ v_2 + w_2 \\ \vdots \\ v_n + w_n \end{bmatrix} \\ &= u_1(v_1 + w_1) + u_2(v_2 + w_2) + \cdots + u_n(v_n + w_n) \end{align*}$$

RHS $= \vec{u}^T\vec{v} + \vec{u}^T\vec{w}$:

$$\begin{align*} \vec{u}^T\vec{v} + \vec{u}^T\vec{w} &= \begin{bmatrix} u_1 & u_2 & \cdots & u_n \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} + \begin{bmatrix} u_1 & u_2 & \cdots & u_n \end{bmatrix} \begin{bmatrix} w_1 \\ w_2 \\ \vdots \\ w_n \end{bmatrix} \\ &= u_1v_1 + u_2v_2 + \cdots + u_nv_n + u_1w_1 + u_2w_2 + \cdots + u_nw_n \\ &= u_1v_1 + u_1w_1 + u_2v_2 + u_2w_2 + \cdots + u_nv_n + u_nw_n \\ &= u_1(v_1 + w_1) + u_2(v_2 + w_2) + \cdots + u_n(v_n + w_n) = \text{LHS} \end{align*}$$

Projections

Setup: Suppose we have a line in $\mathbb{R}^2$ or $\mathbb{R}^3$ & let’s say we have a point off the line. A natural question is: what is the distance from the point to the line, and what is the closest point on the line to our point? Projections give us the tools to answer both.

Although there are many lines from $B$ to $\ell$, we designate the length of the shortest one as the distance from $B$ to $\ell$. The shortest segment from $B$ to $\ell$ must form a 90° angle with $\ell$ - any other angle would produce a longer path.

Let $\vec{u}$ & $\vec{v}$ be two nonzero vectors. Let $\vec{p}$ be the vector obtained by dropping a perpendicular from the tip of $\vec{v}$ onto $\vec{u}$. We call $\vec{p}$ the projection of $\vec{v}$ onto $\vec{u}$.

Two goals:

• Can we find a formula for $\vec{p}$ in terms of $\vec{u}$ & $\vec{v}$?
• Can we find the length of the red line?

Answer to Goal 1

Let $\theta$ be the angle between $\vec{u}$ & $\vec{v}$.

Notice that $\vec{v}$, $\vec{p}$, and $\vec{v} - \vec{p}$ form a right triangle with the right angle at the tip of $\vec{p}$. The angle at the origin is $\theta$, with $\vec{v}$ as the hypotenuse and $\vec{p}$ as the side adjacent to $\theta$. Using $\cos = \text{adjacent}/\text{hypotenuse}$:

$$\cos\theta = \frac{\|\vec{p}\|}{\|\vec{v}\|} \implies \|\vec{p}\| = \|\vec{v}\|\cos\theta \tag{1}$$

Let $\hat{u}$ be the unit vector of $\vec{u}$ (meaning $\hat{u}$ has the same direction as $\vec{u}$, but its length is 1).

$$\hat{u} = \frac{1}{\|\vec{u}\|}\vec{u} \tag{2}$$

Since $\vec{p}$ lies along the line through $\vec{u}$ (it is the foot of the perpendicular from the tip of $\vec{v}$ onto that line), it points in the same direction as $\hat{u}$. Any vector equals its length times its unit direction vector:

$$\vec{p} = \|\vec{p}\|\hat{u} \tag{3}$$

Chaining the Equations Together

By chaining (1), (2), and (3), and recalling from last time that $\cos\theta = \dfrac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|}$:

$$\begin{aligned} \vec{p} &= \|\vec{p}\|\hat{u} && \text{(3)} \\ &= \|\vec{v}\|\cos\theta\; \hat{u} && \text{(1)} \\ &= \|\vec{v}\| \cdot \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|}\; \hat{u} \\ &= \|\vec{v}\| \cdot \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|} \cdot \frac{1}{\|\vec{u}\|}\vec{u} && \text{(2)} \\ &= \cancel{\|\vec{v}\|} \cdot \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\cancel{\|\vec{v}\|}} \cdot \frac{1}{\|\vec{u}\|}\vec{u} \\ &= \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{u}\|}\vec{u} \\ &= \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|^2}\vec{u} && \left(\|\vec{u}\|^2 = \vec{u}^T\vec{u}\right) \\ &= \underbrace{\frac{\vec{u}^T\vec{v}}{\vec{u}^T\vec{u}}}_{\text{scalar}}\,\vec{u} \end{aligned}$$

Definition: Projection

If $\vec{u}$ & $\vec{v}$ are vectors in $\mathbb{R}^n$, & $\vec{u} \neq \vec{0}$, then the projection of $\vec{v}$ onto $\vec{u}$ is denoted by $\text{proj}_{\vec{u}}(\vec{v})$ & is given by:

$$\underbrace{\text{proj}_{\vec{u}}(\vec{v})}_{\vec{p}} = \left(\frac{\vec{u}^T\vec{v}}{\vec{u}^T\vec{u}}\right)\vec{u}$$

Intuition

Intuitively, a projection vector is your “shadow” vector. Picture a light source shining rays perpendicular to $\vec{u}$. The shadow that $\vec{v}$ casts onto the line through $\vec{u}$ is exactly $\vec{p}$.

Note that the shadow always falls on the line through $\vec{u}$ - not just the arrow itself. So $\vec{p}$ can extend beyond the tip of $\vec{u}$ (right diagram) when $\vec{v}$ points more in the direction of $\vec{u}$.

Answer to Goal 2

Q: What is the length of the red segment?

The red segment can be thought of as the vector $\vec{v} - \vec{p}$, so the length we seek is $\|\vec{v} - \vec{p}\|$.

$$\|\vec{v} - \vec{p}\| = \left\|\vec{v} - \text{proj}_{\vec{u}}(\vec{v})\right\| = \left\|\vec{v} - \frac{\vec{u}^T\vec{v}}{\vec{u}^T\vec{u}}\,\vec{u}\right\|$$

Lines & Planes

When it comes to linear equations, we are familiar with the following form $y = mx + b$ (slope-intercept form). We will now describe this via vectors. To do so, we’ll need to get slope-intercept into a different form called the general form of the equation of a line.

General form

$$ax + by = c$$

Slope-intercept form

$$y = mx + b$$

Note: the $b$ in general form and the $b$ in slope-intercept form are different variables.

Converting Between Forms

Example: Convert $7x + 12y = 15$ to slope-intercept form.

$$7x + 12y = 15 \iff 12y = -7x + 15 \iff y = -\frac{7}{12}x + \frac{15}{12}$$

Vector Form of a Line

The goal is to express the equation of a line entirely in terms of vectors, so that the unknowns $x, y$ and the coefficients $a, b$ can each be grouped into a column vector. We do this by systematically renaming variables to use subscript indexing:

$$\begin{aligned} ax + by &= c \\ ax_1 + bx_2 &= c && (x \to x_1,\; y \to x_2) \\ w_1x_1 + w_2x_2 &= c && (a \to w_1,\; b \to w_2) \\ w_1x_1 + w_2x_2 - c &= 0 \\ w_1x_1 + w_2x_2 + b &= 0 && (-c \to b) \end{aligned}$$

This can be written as a matrix product:

$$\begin{bmatrix} w_1 & w_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} + b = 0 \quad\Longrightarrow\quad \vec{w}^T\vec{x} + b = 0 \quad \text{(vector equation of a 2-dimensional line)}$$

If $\vec{w}$ & $\vec{x}$ are 3-dimensional (e.g. $\vec{w} = \begin{bmatrix} 2 \\ 5 \\ 7 \end{bmatrix}$ & $\vec{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}$, & $b = 3$), then $\vec{w}^T\vec{x} + b = 0$ becomes:

$$\begin{bmatrix} 2 & 5 & 7 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} + 3 = 0 \iff 2x_1 + 5x_2 + 7x_3 + 3 = 0 \quad \text{(equation of a plane)}$$

Hyperplanes

• In 2D, $\vec{w}^T\vec{x} + b = 0$ produces a line.
• In 3D, $\vec{w}^T\vec{x} + b = 0$ produces a plane.
• In general (in $\mathbb{R}^n$ or $n$-dimensional plane) $\vec{w}^T\vec{x} + b = 0$ produces a hyperplane.

Hyperplanes sound scary/fancy, but this is just a generalization of a 2D line.

For every hyperplane $\vec{w}^T\vec{x} + b = 0$, the vector $\vec{w}$ is orthogonal to the hyperplane.

Proof

Take any two points $\vec{x}_1$ and $\vec{x}_2$ on the hyperplane. Both satisfy the equation, so $\vec{w}^T\vec{x}_1 + b = 0$ and $\vec{w}^T\vec{x}_2 + b = 0$. Subtracting:

$$\vec{w}^T\vec{x}_2 - \vec{w}^T\vec{x}_1 = 0 \implies \vec{w}^T(\vec{x}_2 - \vec{x}_1) = 0$$

So $\vec{w}$ is orthogonal to any vector $\vec{x}_2 - \vec{x}_1$ that lies in the hyperplane, which is exactly what it means for $\vec{w}$ to be orthogonal to the hyperplane itself.