Linear Space

Linear space is a generalization of $d$-dimensional Euclidean space ($0\leq d\leq 3$) and similar structures. For the relationship between the relevant concepts, see the relationship between Euclidean spaces and linear spaces.

Prerequisites: Abelian groups and fields.

Informally, a set that is closed under an operation and satisfies associativity, has an identity element and inverses forms a group. If it also satisfies commutativity, it forms an Abelian group.

If a set is closed under the four arithmetic operations, it forms a field. See Basic concepts of abstract algebra for the relevant definitions.

Definition

A linear space (vector space) is a fundamental concept and an important object of study in linear algebra. A linear space is a module-like algebraic structure consisting of a set of vectors $V$, a field $\Bbb{P}$, an addition operation $+$, and scalar multiplication.

More specifically, let $(V,+)$ be an Abelian group and let $\Bbb{P}$ be a field.

Define an algebraic operation between elements of $\Bbb{P}$ and elements of $V$, called scalar multiplication: $\cdot:\Bbb{P}\times V\mapsto V$, written as $p\cdot v$ or $pv$, where $p$ belongs to the field $\Bbb{P}$ and $v$ belongs to the Abelian group $V$. This scalar multiplication must be closed: its result must always be meaningful and remain in the group $V$.

It must also satisfy the following conditions:

1. Distributivity of scalar multiplication over vector addition: for $\mathbf u,\mathbf v\in V,a\in \Bbb{P}$, $a(\mathbf u+\mathbf v)=a\mathbf u+a\mathbf v$
2. Distributivity of scalar multiplication over scalar addition: for $a,b\in \Bbb{P},\mathbf u\in V$, $(a+b)\mathbf u=a\mathbf u+b\mathbf u$
3. Associativity of scalar multiplication (consistent with field multiplication): for $a,b\in \Bbb{P},\mathbf u\in V$, $a(b\mathbf u)=(ab)\mathbf u$
4. Multiplicative identity for scalars: let $1\in \Bbb{P}$ be the multiplicative identity of $\Bbb{P}$; then for $u\in V$, $1\mathbf u=\mathbf u$

Then the algebraic system $(V,+,\cdot,\mathbb{P})$ is called a linear space that $V$ forms with $+,\cdot$ over $\Bbb{P}$, $\Bbb{P}$ is called the base field of the linear space, elements of $V$ are called vectors, and elements of $\Bbb{P}$ are called scalars. When $\Bbb{P}$ is the field of real numbers, this is called a real linear space; when $\Bbb{P}$ is the field of complex numbers, it is called a complex linear space.

Whether it is a list of numbers, an arrow, or something else, anything satisfying the axioms above can be regarded as a vector and studied using the theory of linear algebra.

The identity element of the additive group is called the zero vector, written as $\mathbf 0$ or $\mathbf\theta$.

The addition and subtraction of vectors in the original Abelian group, together with the newly defined scalar multiplication in the linear space, are collectively called linear operations.

Note

For convenience, in what follows:

1. Elements of $V$ are not typeset in bold.
2. The algebraic system $(V,+,\cdot,\mathbb{P})$ satisfying the definition of a linear space is also called a linear space.

Be careful to distinguish these usages.

Intuitive understanding

Informally, scalar multiplication corresponds to scaling, and elements of the base field $\Bbb{P}$ represent the ratios of scaling. Vector addition corresponds to superposition. At the same time, elements of $\Bbb{P}$ also represent the range of values of a vector's coordinates.

Conditions 1–4 describe the relationship between scaling and superposition. They can be understood using arrows in the plane.

Basic properties

Note

The following properties can also be found in group theory and related subjects.

For the linear space $(V,+,\cdot,\Bbb{P})$:

1. $\theta$ is unique

2. For $\forall\alpha\in V$, $-\alpha$ is unique

3. $\exists 0\in\mathbb{P}$ such that $\forall\alpha\in V$, $0\alpha=\theta$

4. $\forall k\in\mathbb{P}$, $k\theta=\theta$

5. $(-1)\alpha=-\alpha,~\forall\alpha\in V$

6. No zero divisors: for $\forall\alpha\in V,k\in\mathbb{P}$, $k\alpha=\theta\implies k=0\lor\alpha=\theta$

7. Additive cancellation: for $\forall\alpha,\beta,\gamma\in V$, $\alpha+\beta=\alpha+\gamma\implies\beta=\gamma$

   In fact, additive cancellation is a property of Abelian groups.

Examples

1. $\Bbb{P}^n$ with addition and multiplication over the number field $\Bbb{P}$ forms a linear space over $\Bbb{P}$. For example, $\Bbb{P}$ may be $\Bbb{R}$, $\Bbb{C}$, or $\Bbb{N}_p$ ($p$ is prime).
2. The $n\times m$ matrices over the number field $\Bbb{P}$, namely $\Bbb{P}^{n\times m}$, form a linear space over $\Bbb{P}$ under matrix addition and scalar multiplication.
3. The univariate polynomial ring $\Bbb{P}[x]$ over the number field $\Bbb{P}$ forms a linear space over $\Bbb{P}$ under polynomial addition and scalar multiplication.
4. All continuous functions on the interval $[a,b]$ (denoted $C[a,b]$) form a linear space over the range field under function addition and scalar multiplication of a continuous function by a value.

Related concepts

Linear dependence and linear independence

For the linear space $(V,+,\cdot,\Bbb{P})$:

1. $a_1,a_2,\dots,a_n\in V$ are called a vector group of $V$.
2. For $k_1,k_2,\dots,k_n\in\Bbb{P}$, $\sum_{i=1}^nk_ia_i$ is called a linear combination of the vector group $a_1,a_2,\dots,a_n$.
3. If a vector $\beta\in V$ can be expressed as a linear combination of the vector group $a_1,a_2,\dots,a_n$, then $\beta$ is said to be linearly represented by the vector group $a_1,a_2,\dots,a_n$.
4. For $k_1,k_2,\dots,k_n\in\Bbb{P}$, if the vector group $a_1,a_2,\dots,a_n$ satisfies $\sum_{i=1}^nk_ia_i=\theta\iff k_i=0, i=1,2,\dots,n$, then the vector group $a_1,a_2,\dots,a_n$ is called linearly independent; otherwise the vector group $a_1,a_2,\dots,a_n$ is called linearly dependent.

The zero vector is defined to be linearly dependent with every vector.

A linear representation or linear-dependence equation can be written in matrix multiplication form:

$$\beta=k_1a_1+k_2a_2+\cdots+k_ra_r=(a_1,a_2,\cdots,a_r)\begin{pmatrix} k_1 \\ k_2 \\ \vdots \\ k_r \end{pmatrix}$$

By convention, write the vectors $a$ side by side in order on the left; write the scalars $k$ vertically in order on the right, forming a column vector.

Note that this column vector of scalars is only a convenient formal notation. It is not in the space $V$ and is fundamentally different from the vectors on the left. If the vectors on the left happen to be column vectors, placing them side by side formally forms a matrix, and the product above is exactly the familiar form of a matrix multiplied on the left by a column vector.

As shown below, this linear representation is also equivalent to the vector $\beta$ lying in the image space of the matrix $(a_1,a_2\cdots,a_r)$.

By the definition given below, the zero vector always lies in the image space. From the perspective of linear transformations, linear dependence is equivalent to multiple vectors being mapped to the zero vector, while linear independence is equivalent to only the zero vector itself being mapped to the zero vector.

Properties

For the linear space $(V,+,\cdot,\Bbb{P})$:

1. If part of a vector group is linearly dependent, then the whole vector group is linearly dependent. If a vector group is linearly independent, then every nonempty part of it is linearly independent. In short: “large independent, small independent”; “small dependent, large dependent.”
2. A vector group containing $\theta$ is linearly dependent.
3. A vector group is linearly dependent if and only if one of its vectors can be linearly represented by the remaining vectors.
4. If a vector $\beta$ can be linearly represented by the vector group $a_1,a_2,\dots,a_n$, then this representation is unique if and only if the vector group $a_1,a_2,\dots,a_n$ is linearly independent.
5. If the vector group $a_1,a_2,\dots,a_n$ is linearly independent, then a vector $\beta$ can be linearly represented by the vector group $a_1,a_2,\dots,a_n$ if and only if the vector group $a_1,a_2,\dots,a_n,\beta$ is linearly dependent.

Maximal linearly independent group and rank

Linear dependence can be understood as redundancy: some vectors in a vector group can be represented by the others and can be removed. After removing them, what remains is a maximal linearly independent group.

For the linear space $(V,+,\cdot,\Bbb{P})$:

1. For a vector group $b_1,b_2,\dots,b_m$, let $\{a_1,a_2,\dots,a_n\}\subseteq\{b_1,b_2,\dots,b_m\}$. If:

   • The vector group $a_1,a_2,\dots,a_n$ is linearly independent.
   • For $\forall\beta\in\{b_1,b_2,\dots,b_m\}\setminus\{a_1,a_2,\dots,a_n\}$, the vector group $a_1,a_2,\dots,a_n,\beta$ is linearly dependent.

   Then $a_1,a_2,\dots,a_n$ is called a maximal linearly independent group in the vector group $b_1,b_2,\dots,b_m$. Similarly, a maximal linearly independent group of a linear space $V$ can be defined.

   The maximal linearly independent group of the vector group $\theta,\theta,\dots,\theta$ is defined to be the empty set, so the vector group corresponding to the all-$0$ matrix has no maximal linearly independent group.

   The way of deleting vectors from a vector group is not unique, so the maximal linearly independent group is not unique either. Conventionally, delete them from left to right.

   Coincidentally, the vectors left after deleting them in order are exactly the columns containing the $1$ entries in the row-reduced echelon matrix left by Gaussian elimination from the “view by rows” perspective.

   The size of a maximal linearly independent group of the vector group $b_1,b_2,\dots,b_m$ is called the rank of the vector group, written $\operatorname{rank}\{b_1,b_2,\dots,b_m\}$; define $\operatorname{rank}\{\theta,\theta,\dots,\theta\}=0$.

   Thus, the definition of the rank of a vector group is exactly the same as the definition of the rank of a matrix.

2. If the vector group $a_1,a_2,\dots,a_n$ can linearly represent every vector in the vector group $b_1,b_2,\dots,b_m$, then the vector group $b_1,b_2,\dots,b_m$ is said to be linearly represented by the vector group $a_1,a_2,\dots,a_n$.

3. If the vector group $a_1,a_2,\dots,a_n$ can be linearly represented by the vector group $b_1,b_2,\dots,b_m$, and the vector group $b_1,b_2,\dots,b_m$ can be linearly represented by the vector group $a_1,a_2,\dots,a_n$, then the two vector groups are called equivalent, written $\{a_1,a_2,\dots,a_n\}\cong\{b_1,b_2,\dots,b_m\}$.

   Equivalence of vector groups means that the spaces spanned by the vector groups are the same. Vector groups spanning the same space are equivalent; vector groups spanning different spaces are not equivalent.

   Vector-group equivalence is a stronger condition than matrix equivalence: it requires not only equal rank but also exactly the same space. Therefore, when two matrices are concatenated horizontally, the rank does not change.

   Matrix equivalence only requires equal rank, so matrix equivalence means that the preceding matrix or space can reach the following matrix or space through an invertible transformation.

Properties

For the linear space $(V,+,\cdot,\Bbb{P})$:

1. If the vector group $a_1,a_2,\dots,a_n$ can be linearly represented by the vector group $b_1,b_2,\dots,b_m$:

   • If $n>m$, then the vector group $a_1,a_2,\dots,a_n$ is linearly dependent.
   • If the vector group $a_1,a_2,\dots,a_n$ is linearly independent, then $n\leq m$.

2. Equivalent linearly independent vector groups have the same size.

   Any maximal linearly independent groups of a vector group have the same size.

3. A vector group is linearly independent if and only if its rank equals its size.

4. If the vector group $a_1,a_2,\dots,a_n$ can be linearly represented by the vector group $b_1,b_2,\dots,b_m$, then $\operatorname{rank}\{a_1,a_2,\dots,a_n\}\leq\operatorname{rank}\{b_1,b_2,\dots,b_m\}$.

5. Equivalent vector groups have equal rank.

Linear span

For the linear space $(V,+,\cdot,\Bbb{P})$, $\left\{v=\sum_{i=1}^nk_ia_i:a_i\in V,k_i\in\Bbb{P},i=1,2,\dots,n\right\}$ also forms a linear space, called the linear space spanned by the vector group $a_1,a_2,\dots,a_n$ (or its linear span), written $\operatorname{span}\{a_1,a_2,\dots,a_n\}$.

These $n$ vectors $a$ need not be linearly independent.

Linear subspace

For the linear space $(V,+,\cdot,\Bbb{P})$, if the algebraic system $(V_1,+,\cdot,\Bbb{P})$ satisfies:

1. $\varnothing\ne V_1$
2. $V_1\subseteq V$
3. $V_1$ forms a linear space over $\mathbb{P}$ under $+,\cdot$

then $V_1$ is called a linear subspace of $V$, abbreviated as a subspace, written $V_1\leq V$.

Every space $V$ has two trivial subspaces: itself, $V$, and the zero subspace. The zero subspace contains only the zero vector and no linearly independent vectors.

If $\subseteq$ in condition 2 is replaced by $\subset$, then $V_1$ is called a proper linear subspace of $V$, written $V_1<V$.

It is not difficult to prove that a nonempty subset $V_1$ of a linear space $V$ is a linear subspace if and only if the linear operations are closed on $V_1$, that is:

1. $\forall u,v\in V_1$,$u+v\in V_1$
2. $\forall v\in V_1$,$\forall k\in \Bbb{P}$,$kv\in V_1$

Intersection, sum, direct sum, and direct product

For linear spaces $(V_1,+,\cdot,\Bbb{P})$ and $(V_2,+,\cdot,\Bbb{P})$:

1. It is easy to verify that addition and scalar multiplication are closed on $V_1\cap V_2$, so $V_1\cap V_2$ can be called the intersection of the linear spaces $V_1$ and $V_2$.

   Similarly, the intersection $\bigcap_{i=1}^m V_i$ of multiple linear spaces can be defined.

2. If a linear space $V$ satisfies $V=\{u+v|u\in V_1,v\in V_2\}$, then $V$ is called the sum of the linear spaces $V_1$ and $V_2$, written $V=V_1+V_2$.

   It can be verified that $V_1+V_2$ is the smallest subspace containing $V_1\cup V_2$.

   Similarly, the sum $\sum_{i=1}^m V_i$ of multiple linear spaces can be defined.

3. Let $V=V_1+V_2$. If every element $v$ in the linear space $V$ has exactly one pair of vectors $v_1,v_2$ satisfying $v=v_1+v_2$, then $V$ is called the direct sum of the linear spaces $V_1$ and $V_2$, written $V_1\oplus V_2$.

   Similarly, the direct sum $\bigoplus_{i=1}^m V_i$ of multiple linear spaces can be defined.

4. The direct product $V_1\times V_2$ of $V_1$ and $V_2$ is defined as their Cartesian product equipped with the following addition and scalar multiplication, forming a linear space over $\Bbb{P}$:

   1. $+:(V_1\times V_2)\times(V_1\times V_2)\mapsto V_1\times V_2; ((u_1,v_1),(u_2,v_2))\to (u_1+u_2,v_1+v_2)$
   2. $\cdot:\Bbb{P}\times(V_1\times V_2)\mapsto V_1\times V_2; (k,(u,v))\to (ku,kv)$

   Similarly, the direct product $\prod_{i=1}^m V_i$ of multiple linear spaces can be defined.

Examples

For the linear space $V=\Bbb{R}^3$, let the following be linear spaces:

• $V_1:=\{(x,0,0)|x\in\Bbb{R}\}$
• $V_2:=\{(x,y,0)|x,y\in\Bbb{R}\}$
• $V_3:=\{(0,y,z)|y,z\in\Bbb{R}\}$
• $V_4:=\{(x,0,z)|x,z\in\Bbb{R}\}$

Then:

1. $V_1<V_2<V$,$V_3<V$
2. $V_2=V_1+V_2$
3. $V=V_1\oplus V_3=V_2+V_3$
4. $V_2\oplus V_3=V_4$,$V_2\oplus V_4=V_3$,$V_3\oplus V_4=V_2$
5. $V_2+V_3\leq V$

Properties

1. Let $V_1,V_2,V_3$ be linear spaces over $\Bbb{P}$. Like the intersection of sets, the intersection of linear spaces satisfies:

   1. Commutativity: $V_1\cap V_2=V_2\cap V_1$
   2. Associativity: $V_1\cap(V_2\cap V_3)=(V_1\cap V_2)\cap V_3$

2. Let $V_1,V_2,V_3$ be linear spaces over $\Bbb{P}$. Like the union of sets, the sum of linear spaces satisfies:

   1. Commutativity: $V_1+V_2=V_2+V_1$
   2. Associativity: $V_1+(V_2+V_3)=(V_1+V_2)+V_3$

3. Let $V_1,V_2,V_3$ be linear spaces over $\Bbb{P}$. The intersection and sum of linear spaces satisfy:

   1. $V_1\cap (V_2+V_3)\supseteq (V_1\cap V_2)+(V_1\cap V_3)$
   2. $V_1+(V_2\cap V_3)\subseteq (V_1+V_2)\cap (V_1+V_3)$

4. $\operatorname{span}\{a_1,a_2,\dots,a_n\}+\operatorname{span}\{b_1,b_2,\dots,b_m\}=\operatorname{span}\{a_1,a_2,\dots,a_n,b_1,b_2,\dots,b_m\}$

5. Let $V_1,V_2$ be linear spaces over $\Bbb{P}$. The following statements are equivalent:

   1. $V_1+V_2=V_1\oplus V_2$

   2. $\exists \beta\in V_1+V_2$, such that its decomposition into a sum of vectors from $V_1$ and $V_2$ is unique (any $\to$ exists)

   3. The decomposition of $\theta$ into a sum of vectors from $V_1$ and $V_2$ is unique

   4. $V_1\cap V_2=\{\theta\}$

   ???+ note "Proof" $1\implies 2$: This follows directly from the definition.

   $2 \implies 3$:
   
   Let $\beta=\beta_1+\beta_2$, where $\beta_1\in V_1, \beta_2\in V_2$. If $\theta=\alpha_1+\alpha_2$ and $\theta\ne\alpha_1\in V_1,\alpha_2\in V_2$, then $\beta=\beta+\theta=(\beta_1+\alpha_1)+(\beta_2+\alpha_2)$.
   
   But $\beta_1\ne\beta_1+\alpha_1$, contradicting the condition.
   
   $3 \implies 4$:
   
   Take a nonzero vector $\alpha$ in each of $V_1$ and $V_2$. Then $\theta=\alpha+(-\alpha)=(-\alpha)+\alpha$, contradicting the condition.
   
   $4 \implies 1$:
   
   If $V_1+V_2$ is not a direct sum, then there is a $\beta\in V_1+V_2$ such that $\beta=\beta_1+\beta_2=\gamma_1+\gamma_2$, where $\beta_1,\gamma_1\in V_1,\beta_2,\gamma_2\in V_2$, and $\beta_1,\beta_2,\gamma_1,\gamma_2$ are all distinct.
   
   Thus $\theta\ne\beta_1-\gamma_1=\gamma_2-\beta_2\in V_1\cap V_2$, contradicting the condition.
   

Isomorphism

Let $V,V'$ be linear spaces over the field $\Bbb{P}$. If there is a bijection $\sigma:V\mapsto V'$ that preserves addition and scalar multiplication, that is, for $\forall u,v\in V$,$\forall k\in\Bbb{P}$:

1. $\sigma(u+v)=\sigma(u)+\sigma(v)$
2. $\sigma(ku)=k\sigma(u)$

then $\sigma$ is called an isomorphism from $V$ to $V'$, and $V$ and $V'$ are called isomorphic, written $V\cong V'$.

Note

If $\sigma$ is injective, it is called a monomorphism; if $\sigma$ is surjective, it is called an epimorphism.

Properties

1. Two linear spaces over $\Bbb{P}$ are isomorphic if and only if their dimensions are equal. (See Linear Basis for the definition of dimension.)

2. (A corollary of 1) An $n$-dimensional linear space over $\Bbb{P}$ is isomorphic to the linear space $\Bbb{P}^n$.

   ???+ note "Note" This property shows that, essentially, we can identify coordinates with vectors.

Relationship between Euclidean spaces and linear spaces

Taking the familiar three-dimensional Euclidean space as an example, some corresponding concepts in linear spaces are shown below:

Applications

From this section onward, we mainly discuss the “view by columns” of systems of linear equations.

Matrix $A$ itself is composed of column vectors. Regard $A$ as a group of column vectors, with $x$ as the unknown coefficients, and consider whether these column vectors in $A$ can be combined with the unknowns to form the column vector $b$. Here the column vector $x$ is completely unknown.

The equation $Ax=b$ under consideration can be rearranged as:

$$\alpha_1 x_1 +\alpha_2 x_2 +\cdots+\alpha_n x_n=b$$

In matrix multiplication, the matrix $A$ on the left can be regarded as a group of vectors, namely a group of column vectors. Taking this group of column vectors as a basis, they span a space; we ask whether the column vector $b$ lies in this space.

Solutions of a system of linear equations viewed by columns

Rank is the number of vectors in a maximal linearly independent group and represents the constraints. The remaining vectors then provide the degrees of freedom of the solution, allowing redundant vectors to be assigned in other directions.

If $n$ is the number of columns of matrix A, namely the number of column vectors it contains, and $r(A)$ is the rank of matrix $A$, then the degrees of freedom $S$ are:

$$S=n-r(A)$$

All solutions of the system of equations also form a vector group. The degrees of freedom $S$ are the rank of the solution vector group of $Ax=0$, namely the dimension of the null space discussed below.

Systems with the same solutions

The common solutions of two systems of equations are defined as the intersection of their two solution sets.

Two systems of equations have the same solutions when their solution sets are equal. Systems with equal solution sets have the same solutions; systems with different solution sets do not.

Having the same solutions is also a stronger condition than matrix equivalence: it requires not only equal rank, but also that the rank remain unchanged after the two matrices are concatenated vertically.

In contrast, vector-group equivalence requires the matrices to be concatenated horizontally without changing the rank. Thus, we have the following relationship:

Matrix equivalence does not necessarily imply corresponding vector-group equivalence or systems having the same solutions. However, vector-group equivalence or systems having the same solutions necessarily implies corresponding matrix equivalence (equal rank).

If the vector groups corresponding to two matrices are equivalent, then after transposing the matrices, the corresponding systems have the same solutions, and vice versa.

Null space and image space of a matrix

This discussion of null spaces and image spaces takes the perspective of linear spaces.

For matrix $A$, let $W$ be the set of all solutions $x$ of $Ax=0$. Then $W$ is a linear space, and the scalar field of $W$ is the same as the field containing the entries of $A$.

This $W$ is called the null space of matrix $A$, written $N(A)$.

The null space $N(A)$ of matrix $A$ is exactly the solution space of $Ax=0$. According to the definition of a basis later, a fundamental system of solutions of this equation is a basis of the null space.

If matrix $A$ is invertible, then $A$'s null space $N(A)$ contains only the zero vector.

For matrix $A$, let its $n$ columns be vectors $\alpha$. The space spanned by these $n$ column vectors $\alpha$ is called the image space of $A$, or the column space, written:

$$R(A)=\operatorname{span}\{\alpha_1,\alpha_2,\cdots,\alpha_n\}$$

According to the definition of dimension given later, the dimension of the image space equals the rank of matrix $A$.

By definition, every element $y$ of the image space $R(A)$ has a corresponding representation:

$$y=k_1\alpha_1+k_2\alpha_2+\cdots+k_n\alpha_n=(\alpha_1,\alpha_2,\cdots,\alpha_n)\begin{pmatrix}k_1\\k_2\\\vdots\\k_n\end{pmatrix}=A\begin{pmatrix}k_1\\k_2\\\vdots\\k_n\end{pmatrix}$$

Therefore, the image space $R(A)$ is the range of $Ax$ over all vectors $x$.

Similarly, define the row space of $A$ as the range $R(A^T)$ of the transpose of $A$.

Since the row rank of a matrix equals its column rank, the dimension of the row space is also the rank of the matrix. Thus, transposition changes the image space but not its dimension.

The correspondence with the earlier concepts is:

Vector-group equivalence is equivalent to the corresponding matrices having the same image space $R(A)$.

Systems having the same solutions are equivalent to the corresponding matrices having the same row space $R(A^T)$.

References and notes

1. Qiu Weisheng, Advanced Algebra (Volume II). Tsinghua University Press.
2. Vector space.Wikipedia, The Free Encyclopedia.