Diagonalization

Eigenspaces

All eigenvectors of matrix $A$ belonging to $\lambda_0$, together with the zero vector, form a linear space called an eigenspace of matrix $A$, denoted by $E(\lambda_0)$. It is the homogeneous linear system:

$$(\lambda_0 I-A)X=0$$

This equation describes the solution space.

For the eigenspace $E(\lambda_i)=N(\lambda_i I-A)$, the rank–nullity theorem gives:

$$r(\lambda_i I-A)+\operatorname{dim} N(\lambda_i I-A)=n$$

Therefore, the dimension of the eigenspace $E(\lambda_i)$ is:

$$\operatorname{dim} E(\lambda_i)=n-r(\lambda_i I-A)$$

This is also called the geometric multiplicity of $\lambda_i$.

Invariant Subspaces

When studying a linear transformation $T$, we often want to choose a basis of the space $V$ such that the matrix of $T$ with respect to this basis has the simplest possible form.

Let $V$ be a linear space over the number field $F$, let $W$ be a subspace of $V$, and let $T$ be a linear transformation on $V$. If, for every vector $x$ in $W$, $T(x)$ is also in $W$ (also called invariance or stability of the space under the transformation), then $W$ is called an invariant subspace of $T$.

A space being invariant under a transformation does not mean that its coordinates are literally “unchanged” by the transformation. The space may be stretched or otherwise deformed; it only needs to contain the transformed vectors.

• Every subspace of the linear space $V$ is an invariant subspace of scalar multiplication.
• For any linear transformation $T$ on $V$, the space $V$ and the zero subspace are invariant subspaces of $T$. They are called trivial invariant subspaces.
• The intersection and sum of invariant subspaces are also invariant subspaces.

Let $W$ be an invariant subspace of the linear transformation $T$. Considering only the action of $T$ on $W$ gives a linear transformation on $W$ itself, called the restriction of $T$ to $W$, denoted by ${T|}_W$.

For any linear transformation $T$ on $V$, the range $R(T)$ and the kernel $N(T)$ are invariant subspaces of $T$. These two cases mean that, before and after the transformation, the space $V$ has compressed itself (the range) or has been compressed to $0$ (the kernel).

For any linear transformation $T$ on $V$, the eigenspaces of $T$ are invariant subspaces of $T$.

Primary Decomposition

By the fundamental theorem of algebra, the minimal polynomial can be factored as:

$$m_A(\lambda)={(\lambda-\lambda_1)}^{r_1}\cdots{(\lambda-\lambda_S)}^{r_S}$$

After substituting matrix $A$ for the variable $\lambda$ in the minimal polynomial, the kernels of the individual factors form a series of invariant subspaces of matrix $A$:

$$W_i=N({(\lambda_i I-A)}^{r_i})$$

Theorem: the dimension of the invariant subspace $W_i$ is exactly the algebraic multiplicity of the eigenvalue $\lambda_i$.

Recall that the algebraic multiplicity is the degree of the factor associated with an eigenvalue in the characteristic polynomial, while the geometric multiplicity is the dimension of the eigenspace $E(\lambda_i)=N(\lambda_i I-A)$. The invariant subspace $W_i$ and eigenspace $E(\lambda_i)$ are both kernels of matrices, and those two matrices are related by the $r_i$-th power appearing in the minimal polynomial. In other words, the dimension of the eigenspace is the geometric multiplicity; after the eigenspace is extended through the $r_i$-th power in the minimal polynomial, it reaches an invariant subspace, whose dimension reaches the algebraic multiplicity of the eigenvalue in the characteristic polynomial.

This theorem is actually a corollary of the primary decomposition theorem below.

Let $T$ be the linear transformation corresponding to matrix $A$, and let its restriction to each subspace $W_i$ be $T_i={T|}_{W_i}$. Then the minimal polynomial of $T_i$ is $(x-\lambda_i)^{r_i}$.

Theorem: let $V$ be a linear space over a field $F$, and let $T$ be a linear transformation on $V$. Then $V$ has a primary decomposition with respect to $T$, as a direct sum of several invariant subspaces $W_i$.

$$V=W_1\oplus W_2\oplus\cdots\oplus W_S$$

This means that, with respect to some basis, the matrix of $T$ is a block diagonal matrix:

$$\operatorname{diag}\{A_1,A_2,\cdots,A_S\}$$

Here, $A_i$ is the matrix of $T_i$ in the corresponding basis.

The theorem shows that invariant subspaces can be used to simplify the matrix of a linear transformation.

Diagonalizable Matrices

For an $n$-th order square matrix $A$, if it is similar to a diagonal matrix, then $A$ is called a diagonalizable matrix, or a simple matrix.

• The sum, product, and inverse, when it exists, of diagonal matrices are still diagonal matrices; their diagonal entries are their eigenvalues.
• The matrix of a linear transformation $T$ is diagonalizable if and only if the matrix of $T$ in some basis is diagonal.

Theorem: let the distinct eigenvalues of matrix $A$ be $\lambda_1,\cdots,\lambda_m$. Then the following statements are equivalent:

• Matrix $A$ is diagonalizable.
• Matrix $A$ has $n$ linearly independent eigenvectors.
• The following formula holds:

$$\operatorname{dim} E(\lambda_1)+\cdots+\operatorname{dim} E(\lambda_m)=n$$

As noted earlier, the multiplicity of an eigenvalue in the factorization of the characteristic polynomial is its algebraic multiplicity, and the dimension of its eigenspace is its geometric multiplicity. This theorem also shows that matrix $A$ is diagonalizable if and only if the algebraic and geometric multiplicities of every eigenvalue $\lambda$ of $A$ are equal.

Corollary: if an $n$-th order square matrix $A$ has exactly $n$ distinct eigenvalues, then it must be diagonalizable. The converse does not necessarily hold.

Theorem: matrix $A$ is diagonalizable if and only if the minimal polynomial of $A$ has no repeated roots.

Similarity of matrices also preserves the linear dependence relations among eigenvectors.

Eigenvectors need not be real numbers, and it is entirely possible that one cannot find $n$ linearly independent eigenvectors.

For a repeated eigenvalue, the eigenvectors span a space. To describe this space, representatives must be selected from it. Usually, linearly independent representatives are chosen; their number is the dimension of the space.

When selecting representatives, they are often orthogonalized and normalized. The final result is a set of orthonormal representatives.

Eigenvectors are not necessarily orthogonal, and eigenvectors for different eigenvalues may not be orthogonalizable. Therefore, orthogonalization can only be applied to eigenvectors for a repeated eigenvalue. Normalization, however, can be applied to any eigenvector.

Nilpotent Matrices

Let $T$ be a linear transformation of a space $V$. If there is a positive integer $r$ such that $T^r$ is the zero transformation, then $T$ is called a nilpotent transformation of $V$.

For a positive integer $r$, a matrix satisfying $N^r=0$ is called a nilpotent matrix.

We may further assume that $r$ is the smallest positive integer for which $T^r$ is the zero transformation. Then the minimal polynomial of $T$ is $x^r$, and there is a vector $\xi_0$ such that:

• $$T^r(\xi_0)=0$$
• $$T^{r-1}(\xi_0)\neq 0$$

Cyclic Subspaces

Theorem: let $T$ be a linear transformation of a space $V$, and let $\xi$ be a vector in $V$. If there is a positive integer $s$ such that:

• $$T^s(\xi)=0$$
• $$T^{s-1}(\xi)\neq 0$$

then the vectors $\xi,T(\xi),\cdots,T^{s-1}(\xi)$ are linearly independent.

This theorem gives the following definition:

Let $T$ be a linear transformation of a space $V$, and let $W$ be a subspace of $V$. If there is a vector $\xi_0$ and a positive integer $r$ such that:

• The vectors $\xi_0,T(\xi_0),\cdots,T^{r-1}(\xi_0)$ form a basis of $W$.

• The following equality holds:

  $$T^r(\xi_0)=0$$

then $W$ is called a cyclic subspace with respect to $T$, or simply a $T$-cyclic subspace. Here $\xi_0$ is called a generating vector of the cyclic subspace $W$, and the vectors $\xi_0,T(\xi_0),\cdots,T^{r-1}(\xi_0)$ are called a cyclic basis of $W$.

Clearly, a $T$-cyclic subspace $W$ is invariant under $T$, and every vector $\xi$ in the cyclic subspace $W$ satisfies $T^r(\xi)=0$, where $r$ is the dimension of the cyclic subspace.

Nilpotent Jordan Blocks

If the space $W$ is a cyclic subspace of the transformation $T$, then the restriction ${T|}_W$ of $T$ to $W$ is a nilpotent transformation of $W$. With respect to the cyclic basis of $W$ in reverse order, $T^{r-1}(\xi_0),T^{r-2}(\xi_0),\cdots,\xi_0$, the matrix of ${T|}_W$ is an upper triangular matrix of order $r$ of the following form:

$$N_r=\begin{pmatrix} 0 & 1 & 0 & \cdots & 0 & 0\\ 0 & 0 & 1 & \cdots & 0 & 0\\ 0 & 0 & 0 & \cdots & 0 & 0\\ \vdots & \vdots & \vdots & & \vdots& \vdots\\ 0 & 0 & 0 & \cdots & 0 & 1\\ 0 & 0 & 0 & \cdots & 0 & 0\\ \end{pmatrix}$$

The matrix $N_r$ is called an $r$-th order nilpotent Jordan matrix, or an $r$-th order nilpotent Jordan block.

Let $T$ be a nilpotent transformation of an $n$-dimensional space $V$. The uniquely determined sequence of positive integers $r_1\geq\cdots\geq r_S$ that appears in the decomposition of $V$ into $T$-cyclic subspaces is called the invariant factors of $T$.

For an $n$-th order nilpotent matrix $A$, if $A$ is similar to a matrix $N$ of the form above, it likewise determines a unique sequence of positive integers $r_1\geq\cdots\geq r_S$, called the invariant factors of $A$.

Although a nilpotent matrix cannot be similar to a diagonal matrix, it can be similar to this standard form. In Jordan canonical form, combining similarity diagonalization with the standard form of nilpotent matrices gives the standard form attainable by a general matrix through a similarity transformation.

Some Theorems

1. Let $T$ be a nilpotent transformation of a space $V$, and let

   $$h(x)=a_0+a_1x+\cdots+a_mx^m$$

   be a polynomial. Then the linear transformation $h(T)$ is invertible if and only if $a_0\neq 0$. When $h(T)$ is invertible, $h(T)$'s inverse is also a polynomial in $T$.

2. Let $T$ be a nilpotent transformation of a space $V$, let $W$ be an $r$-dimensional $T$-cyclic subspace, and let $\xi$ be a vector in $W$. If there is an integer $k$ such that

   $$T^{r-k}(\xi)=0$$

   then there is a vector $\eta$ in $W$ such that

   $$\xi=T^k(\eta)$$

3. Let $T$ be a nilpotent transformation of an $n$-dimensional space $V$, and let $x^r$ be the minimal polynomial of $T$. If $W_1$ is an $r$-dimensional $T$-cyclic subspace, then there is a complementary subspace $W_2$ of $W_1$ such that:

   $$V=W_1\oplus W_2$$

   and $W_2$ is also invariant under $T$.

4. Let $T$ be a nilpotent transformation of an $n$-dimensional space $V$. Then $V$ can be decomposed as a direct sum of $T$-cyclic subspaces:

   $$V=W_1\oplus W_2\oplus\cdots\oplus W_S$$

5. Every $n$-th order nilpotent matrix is similar to a matrix of the form:

   $$N=\begin{pmatrix} N_{r_1} & & & 0\\ & N_{r_2} & & \\ & & \cdots & \\ 0 & & & N_{r_S}\\ \end{pmatrix}$$

   where each $N_{r_i}$ is an $r_i$-th order nilpotent Jordan block.

6. If the $T$-cyclic subspaces $W_i$ are ordered by decreasing dimension $r_i$, $r_1\geq\cdots\geq r_S$, then the method of decomposing $V$ into $T$-cyclic subspaces is uniquely determined by $T$.