Lec 04-14-2026: Coordinates and Vectors

Road Map

• Unit 1 - Intro to modeling & G.D.
• Unit 2 - Overview of neural networks
• Unit 3 - Linear Algebra & Support Vector Machines
• Unit 4 - Recommendation Algorithm Overview
• Unit 5 - Simple financial models

Throughout this course, we’ve posed some meaningful questions:

• How does a machine learn?
• How can we find the minimum of a cost function?
• How does text recognition work?
• How can an image be made numeric?
• Can we simplify the notation for a neural network?
• Can we improve our house price model? $$\hat{y}_i = mx_i + b$$ where $x_i$ = square feet

All of the above questions require us to work in a higher-dimensional space of real numbers, and this is a part of Linear Algebra! For us, in modeling, we often observe a phenomenon and the drivers/causes of this phenomenon requires higher dimensions.

To progress any further, we’ll need a crash-course in elementary linear algebra.

Coordinates and Vectors

Coordinates: At first, we may attribute the idea of coordinates to the work in geometry by classic Greek mathematicians (600 BCE - 300 CE).

Geometry takes place in a flat 2D plane or 3D space.

• 2D plane: circles, rectangles
• 3D space: spheres

“Elitist” tools

Greek geometry used only 2 things:

1. Compasses
2. Straight-edge

Amazingly enough, the idea of a coordinate system wasn’t introduced until René Descartes (French mathematician, 1637).

There’s this huge gap because for most of time, algebra & geometry were seen as different fields.

Using a coordinate system allows us to connect geometric objects, like lines, to algebraic objects like equations - a bridge between algebra and geometry that lets us study shapes using equations, and equations using geometry.

Ex.

When we have 2D or 3D, we typically use the $xy$-plane of the $xyz$-coordinate system.

For us, we’ll have many coordinates, so we’ll use the $x_1, x_2, \ldots, x_n$ coordinate system.

Vectors

In the 2D plane, let’s consider the coordinate $(3, 4)$ and draw an arrow from the origin to $(3, 4)$:

• This arrow is called a vector
• The origin is called its initial point
• The tip of the arrow is called its terminal point

How do we denote a vector? We write $\vec{v}$.

By default, we write vectors as column vectors (entries stacked vertically):

$$\vec{v} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}$$

When we want to write a vector in a single row (as a row vector), we use the transpose notation, written $\vec{v}^T$:

$$\vec{v}^T = \begin{bmatrix} 3 & 4 \end{bmatrix}$$

The transpose simply “rotates” the vector from a column to a row (or vice versa). We’ll see the transpose come up often in later computations.

Notice as we change the terminal point, the direction & length change. For this reason, we say a vector is defined by its direction & length (also called its magnitude or norm). Its starting point does not matter.

Loosely speaking, there are 2 types of quantities in math:

Scalar - a single real number

Ex. $77^\circ F$, $5$ meters

Vector - a length (scalar) & a direction

Ex. wind direction $50$ mph SE, displacement (physics)

Without a direction, it’s a scalar.

In $\mathbb{R}^2$, plot $\begin{bmatrix} 7 \\ 0 \end{bmatrix}$, $\begin{bmatrix} 0 \\ 7 \end{bmatrix}$, $\begin{bmatrix} -3 \\ -2 \end{bmatrix}$:

In $\mathbb{R}^3$, plot $\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}$, $\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}$:

Earlier we said vectors are completely characterized by their length & direction - their starting position does not matter.

As long as 2 vectors have the same length & direction, then these vectors are equal, regardless of where they are drawn.

Ex.

In the left diagram, $\vec{v_1}$ and $\vec{v_2}$ point in the same direction and have the same length - they are equal even though they start at different points. In the right diagram, they point in different directions, so they are not equal.

Two vectors are equal provided they are translations of each other (same direction, same length, just shifted).

For convenience, we’ll always make sure our vectors start at the origin. This way, a vector is completely characterized by its terminal point alone - no need to also track the starting point.

Exception

There is one minor exception to a vector being characterized by its length & direction: the zero vector, which has length 0 and therefore no well-defined direction.

Def. The zero vector, denoted by $\vec{0}$, is the vector whose initial & terminal point are the same. It has length 0, and since it doesn’t point anywhere, its direction is undefined.

---

Mathematics follows a consistent pattern: first, we define a structure - a collection of objects we care about. Then we define an algebra on that structure, meaning a set of operations that let us combine and manipulate those objects. This pattern appears throughout all of mathematics:

Structure	Algebra
numbers: $1, 2, 3, 4, \ldots$	$+,\ -,\ \times,\ \div$
functions: $f(x), g(x), h(x), \ldots$	$+,\ -,\ \times,\ \div,\ \circ$ (function compositions: $(f \circ g)(x) = f(g(x))$)
sets	$\cup,\ \cap,\ \subset$
statements	$\land,\ \lor,\ \lnot$

We’ll now define an algebra on vectors - starting with addition.

Vector Addition

The idea is simple: to add two vectors, just add their corresponding components.

Def. Given vectors $\vec{u} = \begin{bmatrix} u_1 \\ u_2 \end{bmatrix}$ & $\vec{v} = \begin{bmatrix} v_1 \\ v_2 \end{bmatrix}$ in $\mathbb{R}^2$, we define

$$\vec{u} + \vec{v} := \begin{bmatrix} u_1 + v_1 \\ u_2 + v_2 \end{bmatrix}$$

That is, add the first components together, and add the second components together (component-wise addition). This extends naturally to higher dimensions - just add each pair of matching components:

Def. Given vectors $\vec{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix}$ & $\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix}$ in $\mathbb{R}^n$, we define

$$\vec{u} + \vec{v} = \begin{bmatrix} u_1 + v_1 \\ u_2 + v_2 \\ \vdots \\ u_n + v_n \end{bmatrix}$$

Note: you can only add two vectors if they have the same number of components (same dimension).

Ex ($\mathbb{R}^2$): If $\vec{u} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$ & $\vec{v} = \begin{bmatrix} 3 \\ 2 \end{bmatrix}$, then

$$\vec{u} + \vec{v} = \begin{bmatrix} 1 + 3 \\ 2 + 2 \end{bmatrix} = \begin{bmatrix} 4 \\ 4 \end{bmatrix}$$

Two ways to geometrically consider this:

1. In each direction, we add the components to get the total horizontal & total vertical displacement.
2. Tail-to-Tip method: Slide $\vec{v}$ so that its starting point sits on the tip of $\vec{u}$. The new arrow drawn from the origin to the tip of the repositioned $\vec{v}$ is $\vec{u} + \vec{v}$. Visually, the three vectors form a triangle.