Lec 04-23-2026: Angles & The Dot Product (cont.)

Recall

Recall that $\vec{u} - \vec{v} = \vec{u} + (-\vec{v})$:

Note

The Pythagorean Theorem only works for right triangles. The triangle formed by $\vec{u}$, $\vec{v}$, and $\vec{u} - \vec{v}$ is generally not a right triangle, so we need a more general tool — the Law of Cosines.

Law of Cosines

Thm (Law of Cosines): In any triangle $ABC$, we have

$$a^2 = b^2 + c^2 - 2bc\cos(A)$$

Proof (acute case)

All angles are less than $90°$. The case for an obtuse triangle is similar.

Suppose $\triangle ABC$ is an acute triangle. Drop a perpendicular from $C$ down to $AB$ — this splits the triangle into two right triangles, letting us use the Pythagorean Theorem on each piece.

By the Pythagorean Theorem, $x^2 + h^2 = b^2 \implies \boxed{h^2 = b^2 - x^2} \qquad \text{(Eq 1)}$

Again by the Pythagorean Theorem, $\boxed{(c - x)^2 + h^2 = a^2} \qquad \text{(Eq 2)}$

Substituting Eq 1 ($h^2 = b^2 - x^2$) into Eq 2:

$$a^2 = (c - x)^2 + h^2 \implies a^2 = (c - x)^2 + b^2 - x^2$$

Expanding $(c - x)^2 = c^2 - 2cx + x^2$ and canceling the $x^2$ terms:

$$\begin{align*} a^2 &= c^2 - 2cx + \cancel{x^2} + b^2 - \cancel{x^2} \\ a^2 &= b^2 + c^2 - 2cx \end{align*}$$

Looking back at our diagram, by SOH CAH TOA: $\cos(A) = \dfrac{x}{b}$, so $\boxed{b\cos(A) = x}$.

SOH CAH TOA

Some Old Horses Can Always Hear Their Owners Approach

Substituting $x = b\cos(A)$:

$$a^2 = b^2 + c^2 - 2bc\cos(A) \qquad \square$$

Let’s now link vector subtraction to the Law of Cosines!

Layer on $\vec{u} - \vec{v}$:

Call the angle between $\vec{u}$ & $\vec{v}$: $\theta$, and study the lengths:

Side Diagram: Subtraction $\vec{u} - \vec{v} = \vec{u} + (-\vec{v})$

$$\vec{u} - \vec{v} = \vec{u} + (-\vec{v})$$ $$\vec{v} = \begin{bmatrix} 3 \\ 1 \end{bmatrix}, \qquad -\vec{v} = \begin{bmatrix} -3 \\ -1 \end{bmatrix}$$

Parallelogram rule

There’s an alternative geometric interpretation: if you place $\vec{u}$ and $\vec{v}$ tail-to-tail, they form two sides of a parallelogram. The vector $\vec{u} - \vec{v}$ is one diagonal of that parallelogram (from the tip of $\vec{v}$ to the tip of $\vec{u}$), while $\vec{u} + \vec{v}$ is the other diagonal.

Thm: If $\theta$ is the angle between two non-zero vectors $\vec{u}$ and $\vec{v}$, where $0 \leq \theta \leq \pi$, then

$$\cos\theta = \frac{u_1v_1 + u_2v_2 + \cdots + u_nv_n}{\|\vec{u}\|\|\vec{v}\|}$$

Proof

Apply the Law of Cosines to the triangle above. The side opposite the angle $\theta$ is $\vec{u} - \vec{v}$, and the two sides forming $\theta$ have lengths $\|\vec{u}\|$ and $\|\vec{v}\|$. Matching to $a^2 = b^2 + c^2 - 2bc\cos(A)$: $a \leftrightarrow \|\vec{u}-\vec{v}\|$, $b \leftrightarrow \|\vec{u}\|$, $c \leftrightarrow \|\vec{v}\|$, $A \leftrightarrow \theta$:

$$\underbrace{\|\vec{u} - \vec{v}\|^2}_{\text{LHS}} = \underbrace{\|\vec{u}\|^2 + \|\vec{v}\|^2 - 2\|\vec{u}\|\|\vec{v}\|\cos\theta}_{\text{RHS}}$$

LHS:

$$\begin{align*} \|\vec{u} - \vec{v}\|^2 &= \left\| \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} - \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} \right\|^2 = \left\| \begin{bmatrix} u_1 - v_1 \\ u_2 - v_2 \\ \vdots \\ u_n - v_n \end{bmatrix} \right\|^2 \\ &= \left(\sqrt{(u_1-v_1)^2 + (u_2-v_2)^2 + \cdots + (u_n-v_n)^2}\right)^2 \\ &= (u_1-v_1)^2 + (u_2-v_2)^2 + \cdots + (u_n-v_n)^2 \\ &\quad\text{(expanding each } (u_i - v_i)^2 = u_i^2 - 2u_iv_i + v_i^2 \text{)} \\ &= u_1^2 - 2u_1v_1 + v_1^2 + u_2^2 - 2u_2v_2 + v_2^2 + \cdots + u_n^2 - 2u_nv_n + v_n^2 \\ &= (u_1^2 + u_2^2 + \cdots + u_n^2) + (v_1^2 + v_2^2 + \cdots + v_n^2) - 2(u_1v_1 + u_2v_2 + \cdots + u_nv_n) \end{align*}$$

RHS:

$$\begin{align*} \|\vec{u}\|^2 + \|\vec{v}\|^2 - 2\|\vec{u}\|\|\vec{v}\|\cos\theta &= \left(\sqrt{u_1^2 + u_2^2 + \cdots + u_n^2}\right)^{\!2} + \left(\sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}\right)^{\!2} - 2\|\vec{u}\|\|\vec{v}\|\cos\theta \\ &= (u_1^2 + u_2^2 + \cdots + u_n^2) + (v_1^2 + v_2^2 + \cdots + v_n^2) - 2\|\vec{u}\|\|\vec{v}\|\cos\theta \end{align*}$$

Setting LHS $=$ RHS and canceling $(u_1^2 + \cdots + u_n^2) + (v_1^2 + \cdots + v_n^2)$ from both sides:

$$-2(u_1v_1 + u_2v_2 + \cdots + u_nv_n) = -2\|\vec{u}\|\|\vec{v}\|\cos\theta$$ $$\implies u_1v_1 + u_2v_2 + \cdots + u_nv_n = \|\vec{u}\|\|\vec{v}\|\cos\theta$$ $$\implies \boxed{\cos\theta = \frac{u_1v_1 + u_2v_2 + \cdots + u_nv_n}{\|\vec{u}\|\|\vec{v}\|}}$$

The Dot Product

The term in the numerator, $u_1v_1 + u_2v_2 + \cdots + u_nv_n$, is a special quantity known as the Dot Product. We denote it by $\vec{u}^T\vec{v}$.

The notation makes sense: $\vec{u}^T$ transposes $\vec{u}$ from a column into a row vector, and then multiplying a row vector by a column vector gives a single number (a scalar) by matching up and summing the products of corresponding entries.

Given $\vec{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix}$ and $\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix}$:

$$\vec{u}^T\vec{v} = \begin{bmatrix} u_1 & u_2 & \cdots & u_n \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} := u_1v_1 + u_2v_2 + \cdots + u_nv_n$$

Neural Net connection

The dot product appears in neural networks: $w_1x_1 + w_2x_2 + \cdots + w_nx_n + b$ (dot product plus bias).

Hence our theorem becomes:

$$\cos\theta = \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|}$$

Note that $\|\vec{u}\|$ and $\|\vec{v}\|$ are always positive for non-zero vectors - squaring each component makes it non-negative, so the square root is always positive. This means the denominator $\|\vec{u}\|\|\vec{v}\|$ is always positive, and therefore the sign of $\vec{u}^T\vec{v}$ determines the sign of $\cos\theta$.

$\|\vec{u}\| = \sqrt{u_1^2 + \cdots + u_n^2}$ and $\|\vec{v}\| = \sqrt{v_1^2 + \cdots + v_n^2}$

Claim: Intuitively, the dot product $\vec{u}^T\vec{v}$ is a measure of directional similarity between $\vec{u}$ and $\vec{v}$ - it tells us how much the two vectors point in the same direction, regardless of where they start.

Song Example: Recommendation (collaborative filtering)

The Setup

Say a music app has three users, A, B, and C, who have each rated some songs on a scale of 1 to 5. The songs are The Message, Normal, and APT. Not every user has listened to every song, so some entries are unknown (marked with $?$):

Each row is a user, each column is a song, and each entry is the rating that user gave that song. The $?$ entries are songs a user hasn’t rated yet — and those are what we want to predict.

Each user’s row of ratings is a vector in $\mathbb{R}^3$ (one dimension per song). For the entries we know:

$$\vec{a} = \begin{bmatrix} 5 \\ ? \\ 4 \end{bmatrix}, \quad \vec{b} = \begin{bmatrix} ? \\ 5 \\ 5 \end{bmatrix}, \quad \vec{c} = \begin{bmatrix} ? \\ ? \\ 5 \end{bmatrix}$$

The $?$ entries are the holes we want to fill. Notice that each row can be thought of as a vector — and if two users’ rating vectors point in a similar direction, their tastes are probably similar. This is where the dot product becomes relevant.

How do we optimize this matrix? That is to say, how do we use those directional similarities to fill in the missing entries?

All Students Take Calculus

All Students Take Calculus — mnemonic for which trig functions are positive in each quadrant.

• Q I ($0$ to $\frac{\pi}{2}$) — All positive: $\sin, \cos, \tan$ all $> 0$
• Q II ($\frac{\pi}{2}$ to $\pi$) — Sine positive: only $\sin > 0$, so $\cos < 0$
• Q III ($\pi$ to $\frac{3\pi}{2}$) — Tangent positive: $\sin$ and $\cos$ both $< 0$
• Q IV ($\frac{3\pi}{2}$ to $2\pi$) — Cosine positive: only $\cos > 0$

In our cases below, $\theta \in [0, \pi]$ only covers Q I and Q II.

Since $\theta \in [0, \pi]$, the sign of $\cos\theta$ — and therefore the sign of $\vec{u}^T\vec{v}$ — depends on where $\theta$ falls. There are 5 cases to consider:

• $\theta = 0$
• $0 < \theta < \dfrac{\pi}{2}$
• $\theta = \dfrac{\pi}{2}$
• $\dfrac{\pi}{2} < \theta < \pi$
• $\theta = \pi$

Case 1: If $\theta = 0$, then the angle between $\vec{u}$ & $\vec{v}$ is 0 — they point in exactly the same direction.

Since $\cos(0) = 1 > 0$ and the denominator $\|\vec{u}\|\|\vec{v}\|$ is always positive:

$$\cos\theta = \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|} \implies \vec{u}^T\vec{v} \text{ is positive.}$$

Case 2: If $0 < \theta < \dfrac{\pi}{2}$, then the angle between $\vec{u}$ & $\vec{v}$ is less than $90°$.

Since $\cos\theta > 0$ for $\theta \in \left(0, \frac{\pi}{2}\right)$ and the denominator is always positive:

$$\cos\theta = \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|} \implies \vec{u}^T\vec{v} \text{ is positive.}$$

Case 3: If $\theta = \dfrac{\pi}{2}$, then the angle between $\vec{u}$ & $\vec{v}$ is $90°$ — they are orthogonal. Orthogonal is the generalization of “perpendicular” to any number of dimensions; in $\mathbb{R}^2$ and $\mathbb{R}^3$ it means exactly the same thing as perpendicular.

Since $\cos\!\left(\tfrac{\pi}{2}\right) = 0$:

$$\cos\theta = \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|} \implies \vec{u}^T\vec{v} = 0$$

Case 4: If $\dfrac{\pi}{2} < \theta < \pi$, then $\vec{u}$ & $\vec{v}$ look like:

So $\vec{u}$ & $\vec{v}$ are pointing in very different directions! Since $\cos\theta < 0$ for $\theta \in \left(\frac{\pi}{2}, \pi\right)$ and the denominator is always positive:

$$\cos\theta = \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|} \implies \vec{u}^T\vec{v} \text{ is NEGATIVE.}$$

Case 5: If $\theta = \pi$, then $\vec{u}$ & $\vec{v}$ point in exactly opposite directions:

Since $\cos(\pi) < 0$ and the denominator is always positive:

$$\cos\theta = \frac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|} \implies \vec{u}^T\vec{v} \text{ is NEGATIVE.}$$

Summary

Since $\cos\theta = \dfrac{\vec{u}^T\vec{v}}{\|\vec{u}\|\|\vec{v}\|}$ and the denominator is always positive, the sign of $\vec{u}^T\vec{v}$ alone tells us the relationship between the two vectors:

• If $\vec{u}^T\vec{v} > 0$, then $\theta \in \left[0, \tfrac{\pi}{2}\right)$ — $\vec{u}$ & $\vec{v}$ point in the same general direction.
• If $\vec{u}^T\vec{v} = 0$, then $\theta = \tfrac{\pi}{2}$ — $\vec{u}$ & $\vec{v}$ are orthogonal (perpendicular).
• If $\vec{u}^T\vec{v} < 0$, then $\theta \in \left(\tfrac{\pi}{2}, \pi\right]$ — $\vec{u}$ & $\vec{v}$ point in opposite general directions.