Chapter 2: Matrices and Linear Transformations#

Think of a matrix not as a dusty rectangle of numbers, but as a machine that takes a vector, twists and stretches it, and hands you a new vector. In this chapter we’ll wire up that machine, learn to read its blueprints, and discover four different ways to combine two machines into one. This is where linear algebra starts to feel like a superpower.

The Big Picture#

A matrix is a neat way to write down a linear transformation — a rule that respects addition and scaling. Once you see that a matrix‑vector product is just a combination of the matrix’s columns, the whole subject opens up. We’ll then explore the geometry of solving with the row picture (intersecting lines or planes) and the column picture (reaching with the columns of ). After that we’ll build matrix‑matrix multiplication from four equivalent mental models, and finally learn how to split large matrices into blocks and multiply them block‑by‑block. Every idea here will be used again and again when we solve systems, compute inverses, and work with vector spaces.

A Matrix as a Linear Map#

A matrix is an array of numbers:

We say has rows and columns. If the matrix is square.

A vector with components is a column of numbers. For example, lives in (real ‑dimensional space).

Linear transformation: A rule that takes a vector in to a vector in and satisfies, for any vectors and any scalar :

Any matrix defines a linear transformation , and every linear transformation from to can be represented by a unique matrix. The columns of are simply what you get when you apply the transformation to the standard basis vectors: is the ‑th column of . That fact is the key to everything that follows.

📝 Section Recap: A matrix is a rectangular table of numbers that represents a linear transformation; the transformation is completely determined by where it sends the standard basis vectors, and those images are simply the matrix’s columns.

Multiplying a Matrix by a Vector#

Let be and an vector. The product is an vector. There are two natural ways to compute it.

1. The row rule (dot‑product view). The ‑th component of is the dot product of the ‑th row of with :

This is just the usual “row times column” operation you learned with numbers. It’s fast for hand calculation but hides the geometry.

2. The column rule (linear combination view). Write as a list of its columns:

Then

The product is a combination of the columns of , with the entries of telling you how much of each column to use. This shows right away that lies in the span of the columns — the set of all possible combinations, which we will later call the column space.

Example. Let

Row rule: ; , so . Column rule: , , so .

Notice how the column rule mirrors the linear transformation idea: the output is just a blend of the two columns. Because is built from columns, we immediately see properties like and , confirming that really is a linear transformation.

📝 Section Recap: can be computed with dot products of rows with (row rule) or as a weighted sum of the columns of (column rule). The column rule reveals the transformation’s linear nature.

The Geometry of : Row and Column Views#

When we try to solve — find an that makes the product equal a given vector — we can look at the problem in two complementary geometric ways.

The row picture: intersecting hyperplanes#

Each row of gives one linear equation. For a system with unknowns and equations, each equation describes a flat hyperplane in . (In two dimensions, a hyperplane is a line; in three dimensions, it’s a plane.) The solution set of the whole system is the intersection of all hyperplanes — the set of points that satisfy every equation at once.

With unknowns, we usually need independent equations to get a single intersection point. For example,

corresponds to two lines in the plane. Their intersection is the single point , because plugging in gives and . If the lines are parallel, there is no intersection; if they are the same line, there are infinitely many solutions.

In higher dimensions, the row picture remains a story about hyperplanes — but visualizing intersection of many planes gets tricky. That’s why we also need the column picture.

The column picture: reaching with the columns of #

Instead of thinking about intersecting hyperplanes, ask: can we choose weights so that the combination of the columns of exactly equals ? That is,

The equation has a solution precisely when is in the span of the columns — that is, when can be built from the column vectors.

For the example above, the two columns are and . We want weights so that . The column picture asks: how many steps along and do we take to land on ? The same arithmetic shows works: copy of the first column plus copies of the second gives .

If happens to be in the same direction as one of the columns, the weight on that column is obvious. If lies outside the plane (or line) spanned by the columns, no combination works and the system has no solution.

📝 Section Recap: The row picture views as the intersection of hyperplanes, while the column picture views it as building out of the columns of using the weights .

Multiplying Matrices: #

Now we move from multiplying a matrix by a vector to multiplying two matrices. Suppose is and is . Their product is an matrix. There are four equivalent ways to think about this product, each helpful in different situations.

1. Dot‑product definition (the standard rule)#

The entry in row , column of is the dot product of the -th row of with the -th column of :

This is the definition you probably learned first: “row times column”. For example, with

we get

It’s mechanical and always works, but it doesn’t reveal structure. The next three views open up that structure.

2. Column‑wise multiplication: times each column of #

Write as a row of columns: . Then

Each column of is simply applied to the corresponding column of . Because we already understand as a combination of the columns of , this tells us the columns of are all combinations of the columns of . In other words, the column space of (the set of all combinations of its columns) is contained in the column space of .

For our example:

exactly the columns of .

3. Row‑wise multiplication: each row of times #

Write as a column of rows: . Then

where each is a row vector (since is and is ). Each row of is a combination of the rows of — the weights are the entries of the row of . So the row space of is contained in the row space of .

4. Column‑row multiplication: sum of outer products#

This is the most surprising view and a powerful thinking tool. Split into columns and into rows:

where each is an -vector and each is a row vector (the -th row of ). Then

What is ? It’s an matrix — the outer product of a column and a row. Its entry is , exactly the contribution of the -th term to the final . So the product is the sum of rank‑1 matrices (matrices that are just a column times a row), each built from one column of and the corresponding row of . This viewpoint explains why often reveals underlying low‑rank structure (the matrix can be built from a small number of simple pieces).

Using our example:

Add them: .

📝 Section Recap: Matrix multiplication can be carried out by dot products of rows and columns, by multiplying by each column of , by multiplying each row of by , or as a sum of outer products of columns of with the matching rows of . All four methods are equivalent and provide different insights into the structure of the product.

Block Multiplication#

When matrices become large, we often partition them into smaller blocks (submatrices) and multiply as if the blocks were ordinary numbers, provided the partitions conform.

Suppose we split and into blocks like this:

We can multiply them block‑by‑block just as with a matrix product:

as long as all the products on the right make sense. That means the column count of must match the row count of , the column count of matches the row count of , and so on. In other words, the blocks must be partitioned conformably.

Example. Let

Partition both after the second row and second column:

Then

Compute each block using ordinary matrix multiplication (they are all conformable):

• ,
• , so the first block sum is .
• , , sum to .
• , , sum to .
• , , sum to .

The resulting full matrix is

which you can verify by direct multiplication.

Block multiplication is extremely useful for deriving fast algorithms, exploiting matrix structure (like block‑diagonal or block‑triangular forms), and for parallel computing.

📝 Section Recap: When matrices are partitioned into conformable blocks, the product can be computed block‑by‑block using the same rules as scalar matrix multiplication. This lets us handle large structured matrices in a modular way.

Summary#

You now have a whole toolkit of ways to think about matrices and their products. A matrix is not just a table — it’s a linear transformation, and multiplying it by a vector builds that vector out of the matrix’s columns. Solving can be seen as finding the intersection of hyperplanes (row picture) or as reaching with the columns of (column picture). When you multiply two matrices, you can do it with dot products, column by column, row by row, or as a sum of outer products — each perspective shines a different light on the structure of the product. Finally, block multiplication scales these ideas to large matrices by treating submatrices as building blocks. All these viewpoints will be your trusty companions as we move to solving linear systems and understanding vector spaces.