Null Space of the Transpose (The Left Null Space)

In the study of linear algebra, a matrix is often conceptualized as a linear transformation, a process that maps input vectors to output vectors. The set of vectors that this transformation maps to zero, known as the null space, reveals crucial information about the transformation's kernel. However, this perspective provides only part of the picture. Every matrix has a transpose, which represents a related but distinct transformation. The critical question then arises: what happens when we consider the null space of this transposed matrix? This inquiry leads us to the concept of the ​​left null space​​, a fundamental subspace with profound implications.

This article addresses the role and significance of the left null space, a concept often seen as more abstract than its counterparts. We will demystify this essential component of a matrix's structure and illustrate its power. Over the following chapters, you will gain a comprehensive understanding of this topic. The "Principles and Mechanisms" chapter will define the left null space, explore its properties as a vector subspace, establish its dimensional relationship with a matrix's rank, and uncover its beautiful geometric orthogonality to the column space. Subsequently, the "Applications and Interdisciplinary Connections" chapter will demonstrate how this seemingly theoretical idea is the ultimate arbiter of solvability for linear systems, the foundation for least-squares approximations in data science, and a unifying concept in fields ranging from network analysis to numerical computation.

In our journey through linear algebra, we often think of a matrix $A$ as an operator, a machine that takes an input vector $\mathbf{x}$ and transforms it into an output vector $A\mathbf{x}$. A particularly interesting question is: which vectors are completely "crushed" by this machine, transformed into the zero vector? This collection of vectors forms the ​​null space​​, $N(A)$. It's a fundamental concept, telling us about the kernel of the transformation, the vectors that lose their identity in the process.

But this is only half the story. The matrix $A$ has an alter ego, its transpose $A^T$, which you can picture as the same machine but with its internal wiring reversed. What happens if we ask the same question about this transposed matrix? What vectors does it crush to zero? The answer leads us to a second, equally important space: the ​​left null space​​ of the original matrix $A$.

The left null space of a matrix $A$ is, by definition, the null space of its transpose, $A^T$. It is the set of all vectors $\mathbf{y}$ for which $A^T \mathbf{y} = \mathbf{0}$. You might wonder about the name "left null space". If we take the transpose of the entire equation, we get $(\mathbf{y}^T A)^T = \mathbf{0}^T$, which simplifies to $\mathbf{y}^T A = \mathbf{0}^T$. This reveals the origin of the name: it's the space of vectors that, when placed on the left of $A$ (as a row vector), annihilate the matrix, yielding a row of zeros.

Before we dive deeper, let's get a feel for where these vectors live. If our original matrix $A$ has $m$ rows and $n$ columns (an $m \times n$ matrix), its transpose $A^T$ will have $n$ rows and $m$ columns. For the multiplication $A^T \mathbf{y}$ to be defined, the vector $\mathbf{y}$ must have a number of components equal to the number of columns in $A^T$, which is $m$. So, for an $m \times n$ matrix $A$, its left null space $N(A^T)$ is always a subspace of $\mathbb{R}^m$. The familiar null space $N(A)$, on the other hand, lives in $\mathbb{R}^n$. This is a crucial distinction: these two null spaces generally live in entirely different universes, unless the matrix happens to be square ($m=n$).

Let's make this concrete. Consider the matrix:

This is a $3 \times 2$ matrix, so its left null space will be a subspace of $\mathbb{R}^3$. To find it, we first find its transpose:

Now we seek all vectors $\mathbf{y} = \begin{pmatrix} y_1 \\ y_2 \\ y_3 \end{pmatrix}$ such that $A^T \mathbf{y} = \mathbf{0}$. This gives us a system of linear equations:

From the second equation, we find $y_2 = -2y_3$. Substituting this into the first equation gives $y_1 + 2(-2y_3) + 3y_3 = 0$, which simplifies to $y_1 - y_3 = 0$, or $y_1 = y_3$. So, any vector in this space must look like $\begin{pmatrix} y_3 \\ -2y_3 \\ y_3 \end{pmatrix}$. We can factor out the free variable $y_3$ to see the underlying structure: $y_3 \begin{pmatrix} 1 \\ -2 \\ 1 \end{pmatrix}$. This means the left null space of $A$ is a one-dimensional line in $\mathbb{R}^3$ spanned by the single basis vector $\begin{pmatrix} 1 \\ -2 \\ 1 \end{pmatrix}$.

We call it a "space" for a very good reason. It's not just a random collection of vectors; it has a robust structure. If you find two vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ in the left null space, then any linear combination of them, like $c_1\mathbf{v}_1 + c_2\mathbf{v}_2$, will also be in the left null space.

Why? The logic is simple and elegant. If $\mathbf{v}_1$ and $\mathbf{v}_2$ are in $N(A^T)$, it means $A^T \mathbf{v}_1 = \mathbf{0}$ and $A^T \mathbf{v}_2 = \mathbf{0}$. Let's test their sum:

It works! The sum is also in the space. The same holds for scalar multiples. This property of closure under addition and scalar multiplication is the very definition of a vector subspace. This predictable structure is what makes these spaces so powerful in our analysis.

So, how large is this left null space? Its dimension isn't arbitrary. It's intimately tied to the other fundamental properties of the matrix through a beautiful relationship.

You may recall the ​​Rank-Nullity Theorem​​, which for any $m \times n$ matrix $A$ states:

We can apply this very same theorem to the transpose, $A^T$. Since $A^T$ is an $n \times m$ matrix, the theorem tells us:

Here comes the magic ingredient: a cornerstone of linear algebra is that a matrix and its transpose have the same rank. That is, $\operatorname{rank}(A) = \operatorname{rank}(A^T)$. This deep fact connects the span of the columns to the span of the rows. Substituting this into our equation for $A^T$, we get the master equation for the left null space:

This is a profound statement. It means the dimension of the left null space is completely determined by the rank of the matrix and the number of rows. If a $5 \times 7$ matrix $A$ has a column space of dimension 4 (meaning its rank is 4), we immediately know the dimension of its left null space must be $5 - 4 = 1$. If a $5 \times 3$ matrix has a row space of dimension 2 (rank 2), its left null space must have dimension $5-2=3$.

These dimensional relationships are locked together in a kind of cosmic balance sheet. For any $m \times n$ matrix with rank $r$, we have:

• $\dim(N(A)) = n - r$
• $\dim(N(A^T)) = m - r$

If you're told that for a $7 \times 10$ matrix, the dimensions of the two null spaces add up to 9, you can deduce the rank must be 4, and from that, the dimension of the column space is also 4.

There is a beautiful geometric interpretation of the left null space. Remember that a vector $\mathbf{y}$ is in $N(A^T)$ if $\mathbf{y}^T A = \mathbf{0}^T$. Let's write $A$ in terms of its columns: $A = \begin{pmatrix} | & & | \\ \mathbf{c}_1 & \dots & \mathbf{c}_n \\ | & & | \end{pmatrix}$. The equation $\mathbf{y}^T A = \mathbf{0}^T$ means that the dot product of $\mathbf{y}$ with every single column of $A$ is zero: $\mathbf{y} \cdot \mathbf{c}_i = 0$ for all $i$.

This means every vector in the left null space is ​​orthogonal​​ to every column of $A$. And since the columns of $A$ span the ​​column space​​ $C(A)$, it follows that every vector in the left null space is orthogonal to the entire column space.

So, here is the grand picture: In the universe of $\mathbb{R}^m$, the column space $C(A)$ and the left null space $N(A^T)$ exist as orthogonal subspaces. They meet only at the origin, and they are perpendicular to each other in every possible direction. This is why their dimensions must sum to the dimension of the whole space: $\dim(C(A)) + \dim(N(A^T)) = m$.

This orthogonality is not just an abstract curiosity; it is the key to understanding when systems of linear equations have solutions. The system $A\mathbf{x} = \mathbf{b}$ has a solution if and only if the vector $\mathbf{b}$ lies in the column space of $A$. The orthogonality condition gives us a perfect test for this: $\mathbf{b}$ is in the column space if and only if it is orthogonal to every vector in the left null space. This powerful idea is known as the ​​Fredholm Alternative​​. If you have a system of equations that seems unsolvable, you can check if the right-hand side vector $\mathbf{b}$ is "perpendicular" to the "problem-causing" directions defined by the left null space.

The connections we've uncovered also work in reverse. Knowing the properties of a matrix's fundamental subspaces allows us to deduce properties of the matrix itself. If you're told that the left null space of a $3 \times 2$ matrix $A$ is spanned by the vector $\mathbf{v} = [1, 0, -1]^T$, this isn't just abstract information. It's a concrete blueprint.

The condition means that $\mathbf{v}^T A = \mathbf{0}^T$. If the rows of $A$ are $\mathbf{r}_1, \mathbf{r}_2, \mathbf{r}_3$, this translates to:

This gives us a structural constraint on the matrix: its first row must be identical to its third row, $\mathbf{r}_1 = \mathbf{r}_3$. Abstract properties of a subspace dictate concrete algebraic relationships among the entries of the matrix.

And as a final thought, what happens when things get weird? In the familiar world of real vectors, a non-zero vector can never be orthogonal to itself. But in more exotic settings, like vector spaces over complex numbers, it's possible for a non-zero vector $\mathbf{v}$ to satisfy $\mathbf{v}^T \mathbf{v} = 0$. In such a scenario, if we construct a matrix $A = \mathbf{v}\mathbf{u}^T$, its column space is simply the line spanned by $\mathbf{v}$. The left null space is the set of all vectors orthogonal to $\mathbf{v}$. But since $\mathbf{v}$ is orthogonal to itself, the entire column space lies inside the left null space!. This is a beautiful, counter-intuitive result that reminds us that the principles we've discussed are gateways to even richer and more fascinating mathematical structures. The humble left null space is not just a calculation to be performed; it is a key that unlocks a deeper understanding of the symmetry and geometry hidden within matrices.

In our journey so far, we have become acquainted with the cast of characters that populate the world of a matrix—the four fundamental subspaces. One of these, the left null space, might have seemed a bit more mysterious than the others. It is the set of all vectors $\mathbf{y}$ that, when multiplied by our matrix $A$ from the left as $\mathbf{y}^T A$, produce nothing but a row of zeros. What is the point of such a thing? It turns out this seemingly obscure space is not a minor character at all; it is the ultimate arbiter, the supreme judge that decides some of the most fundamental questions in science and engineering. Its properties echo in fields as diverse as data analysis, computer graphics, and even the study of complex networks. Let us now see this powerful idea in action.

Imagine you are trying to achieve a certain outcome, represented by a vector $\mathbf{b}$. Your tools for getting there are a set of linear processes, encapsulated in a matrix $A$. The question "Can I achieve outcome $\mathbf{b}$?" is mathematically phrased as "Does the system $A\mathbf{x}=\mathbf{b}$ have a solution?". You can think of the columns of $A$ as your available ingredients, and the column space, $\text{Col}(A)$, as the collection of all possible dishes you can prepare by mixing them. The system has a solution if and only if your desired dish, $\mathbf{b}$, is on the menu—that is, if $\mathbf{b}$ lies in $\text{Col}(A)$.

But how do you check this without trying every possible combination? This is where the left null space, $N(A^T)$, plays its decisive role. You see, the left null space is the orthogonal complement of the column space. It represents a set of "anti-recipes"—directions fundamentally incompatible with your ingredients. If your target vector $\mathbf{b}$ has any projection onto this "anti-recipe" space, it's impossible to create. The test is beautifully simple: a solution exists if and only if $\mathbf{b}$ is orthogonal to every vector in the left null space.

Therefore, to prove a system is inconsistent, you don't need to check every vector in $N(A^T)$. You just need to find one vector $\mathbf{y}$ in the left null space for which the dot product $\mathbf{y}^T \mathbf{b}$ is not zero. If you find such a vector, the case is closed: no solution exists. This principle, a form of the Fredholm alternative, gives us a concrete condition for solvability. Instead of an exhaustive search, we can characterize all impossible outcomes by finding a basis for $N(A^T)$. Any target $\mathbf{b}$ that is not perpendicular to these basis vectors is unreachable. From a higher viewpoint, this tells us that a linear transformation is surjective (it can reach every point in its target space) precisely when the only vector orthogonal to its image is the zero vector—that is, when its left null space is trivial.

So, what do we do when the judge declares our system "unsolvable"? Do we simply give up? In the real world, this happens all the time. Our measurements are noisy, our models are imperfect, and we often have more data points than parameters in our model. This leads to overdetermined systems $A\mathbf{x}=\mathbf{b}$ that have no exact solution. The vector $\mathbf{b}$ of our measurements simply does not lie in the column space of our model matrix $A$.

Here, linear algebra offers not a surrender, but a beautiful compromise: the least-squares solution. If we can't land exactly on the target $\mathbf{b}$, we can find the point $\mathbf{p}$ inside the column space $\text{Col}(A)$ that is closest to $\mathbf{b}$. This point $\mathbf{p}$ is our best possible approximation, and it is of the form $\mathbf{p} = A\hat{\mathbf{x}}$ for some vector $\hat{\mathbf{x}}$, which we call the least-squares solution.

What does "closest" mean geometrically? It means that the error vector, the difference between our data and our best approximation, $\mathbf{e} = \mathbf{b} - \mathbf{p}$, must be as short as possible. This happens when $\mathbf{e}$ is perpendicular to the space we are projecting onto, $\text{Col}(A)$. But we have just seen that the space of all vectors perpendicular to $\text{Col}(A)$ is none other than the left null space, $N(A^T)$! So, the profound condition that defines the best possible approximation is that the error vector must live in the left null space: $\mathbf{e} \in N(A^T)$. This simple geometric fact is the heart of regression analysis, data fitting, and countless optimization problems. And should we be so lucky that our least-squares error turns out to be zero, it means our error vector is the zero vector. This implies our data $\mathbf{b}$ was in the column space all along, and our system had a perfect solution waiting to be discovered.

It is one thing to appreciate these beautiful geometric relationships, but quite another to compute these subspaces for a giant matrix with millions of entries. Fortunately, the architects of numerical linear algebra have given us powerful tools that act like X-rays for matrices, revealing their internal structure, including the left null space.

Two of the most important tools are the QR factorization and the Singular Value Decomposition (SVD).

When we perform a full QR factorization on an $m \times n$ matrix $A$ (with $m \gt n$), we decompose it as $A=QR$. Here, $Q$ is an $m \times m$ orthogonal matrix whose columns form an orthonormal basis for the entire space $\mathbb{R}^m$, and $R$ is an $m \times n$ upper trapezoidal matrix. The first $n$ columns of $Q$ are constructed to form a pristine orthonormal basis for the column space of $A$. What about the remaining $m-n$ columns of $Q$? By the very nature of an orthogonal matrix, they are orthogonal to the first $n$ columns. They therefore form a perfect orthonormal basis for the orthogonal complement of the column space—that is, for the left null space, $N(A^T)$.

The Singular Value Decomposition, $A = U\Sigma V^T$, is even more revealing. It simultaneously provides orthonormal bases for all four fundamental subspaces. For our purposes, the key is the $m \times m$ orthogonal matrix $U$. The first $r$ columns of $U$ (where $r$ is the rank of $A$) span the column space. The remaining $m-r$ columns of $U$ give us an orthonormal basis for the left null space. The SVD even tells us the dimension of this space directly: it is simply the number of all-zero rows in the central matrix $\Sigma$. These decompositions are not mere theoretical curiosities; they are the robust, high-performance engines running inside the software we use for everything from weather prediction to designing aircraft.

The true beauty of a deep mathematical concept is that it refuses to be confined to its original context. The left null space is a prime example, appearing in surprising places with profound physical and structural interpretations.

Consider a directed graph, like a network of one-way streets or electrical circuits. We can describe its topology with a vertex-edge "incidence matrix" $M$. Let's assign a scalar value—a "potential," like voltage or altitude—to each vertex in the graph, forming a vector $\mathbf{p}$. What does it mean if this vector $\mathbf{p}$ lies in the left null space of the incidence matrix, $M^T \mathbf{p} = \mathbf{0}$? The equation $M^T \mathbf{p} = \mathbf{0}$ unpacks into a simple condition for every single edge in the graph: if an edge runs from vertex $v_i$ to vertex $v_j$, then the potentials must be equal, $p_i = p_j$. This implies that the potential must be constant across any connected component of the graph. The dimension of the left null space, therefore, counts something tangible: the number of separate, weakly connected components in our network!. This single algebraic idea unifies concepts from circuit theory (Kirchhoff's Voltage Law) and graph theory.

This principle of generality doesn't stop with vectors of numbers. The concepts of linear algebra apply just as well to vector spaces of functions, such as the space of polynomials $\mathcal{P}_2(\mathbb{R})$. We can define linear operators on this space, for example, an operator that involves differentiation. Such an operator can be represented by a matrix $A$ with respect to a basis (like $\{1, x, x^2\}$). Finding the left null space of this matrix $A$ reveals fundamental properties of the operator itself—it tells us about the "constraints" on the outputs it can produce.

From a simple question of solvability, we have journeyed to the heart of approximation theory, peered into the machinery of modern computation, and found echoes of the same idea in the structure of networks and functions. The left null space is a testament to the remarkable unity of mathematics, where a single, elegant concept can provide the key to understanding and solving a vast array of problems across the scientific landscape.