Study Guides / Linear Algebra / Exercise session 4
EXERCISE SESSION 4 · ETH ZURICH · HS19 · linalg

Invertible and Orthogonal Matrices

Invertible matrices, elementary matrices, orthogonal/unitary matrices

SB
Steven Battilana Quant · ex-ETH

Invertible Matrices

Definition

A matrix A∈En×nA \in \mathbb{E}^{n \times n} is called invertible
⟺∃!B∈En×n\Longleftrightarrow \exists !B \in \mathbb{E}^{n \times n} such that AB=BA=𝟙AB = BA = \text{𝟙}.

𝟙:=(10⋯00⋱⋱⋮⋮⋱⋱00⋯01)=^identity matrix\text{𝟙} := \begin{pmatrix} 1 & 0 & \cdots & 0\\ 0 & \ddots & \ddots & \vdots\\ \vdots & \ddots & \ddots & 0\\ 0 & \cdots & 0 & 1 \end{pmatrix} \quad \mathrel{\hat{=}} \text{identity matrix}

BB is then called the inverse of AA and is usually denoted by B=A−1B=A^{-1}.

Remark properties

Let A,B∈En×nA,B \in \mathbb{E}^{n \times n}; then the following hold:

(i) If A−1A^{-1} exists, then A−1A^{-1} is unique
(ii) (A−1)−1=A(A^{-1})^{-1} = A
(iii) (AB)−1=B−1A−1(AB)^{-1} = B^{-1}A^{-1}
(iv) (AH)−1=(A−1)H(A^H)^{-1} = (A^{-1})^H
(v) (AT)−1=(A−1)T(A^T)^{-1} = (A^{-1})^T

Definition

For a 2×22 \times 2 matrix you can use the following definition to compute the inverse.

A−1=(abcd)−1=1det⁡(A)(d−b−ca)=1ad−bc(d−b−ca).\begin{aligned} A^{-1} &= \begin{pmatrix} a & b\\ c & d \end{pmatrix}^{-1}\\ &= \frac{1}{\det(A)} \begin{pmatrix} d & -b\\ -c & a \end{pmatrix}\\ &= \frac{1}{ad - bc} \begin{pmatrix} d & -b\\ -c & a \end{pmatrix}. \end{aligned}
Definition

A matrix A∈En×nA \in \mathbb{E}^{n \times n} is called symmetric/Hermitian if: A=AH.\quad A = A^H.

Remark

Let A,B∈En×nA,B \in \mathbb{E}^{n \times n}; then
(AB)H=BHAH(AB)^{H} = B^{H}A^{H} (or (AB)T=BTAT(AB)^{T} = B^{T}A^{T}, respectively).

Satz

Let A∈En×nA \in \mathbb{E}^{n \times n}; then the following hold:

A is invertible ⟺A is regular⟺∀b∈En,  linear system Ax=b:∃!x∈En s.t.x=A−1b⟺rank(A)=n ("A has full rank")\begin{aligned} A \text{ is invertible } &\Longleftrightarrow A \text{ is regular}\\ &\Longleftrightarrow \forall b \in \mathbb{E}^n,\,\, \text{linear system } Ax=b: \quad \exists! x \in \mathbb{E}^n \text{ s.t.}\quad x=A^{-1}b\\ &\Longleftrightarrow \text{rank}(A)=n \quad \text{ ("}A\text{ has full rank")} \end{aligned}
Definition

If mm is an arbitrary natural number, 1≤i,j≤m1\leq i,j \leq m with i≠j, λ∈Ei\neq j,\, \lambda \in \mathbb{E} and A∈Em×mA \in \mathbb{E}^{m \times m}, then the matrices Pij,Si(λ),Eij(λ)∈Em×mP_{ij}, S_i(\lambda), E_{ij}(\lambda) \in \mathbb{E}^{m \times m} are called elementary matrices.

(i) To swap row ii with row jj, multiply from the left by the permutation matrix: PijAP_{ij}A

Pij=⋱⋱⋱i-th column⋱⋱⋱⋱j-th column⋱⋱⋱⋱(⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱1⋱⋱⋱1⋱0⋱1⋱1⋱⋱⋱1⋱1⋱0⋱1⋱⋱⋱1⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱)⋱⋱⋱⋱i-th row⋱⋱⋱⋱j-th row⋱⋱⋱P_{ij} = \begin{array}{cccl} & \begin{array}{ccccccccccc} \phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \mathclap{\scriptsize\text{$i$-th column}}\phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \mathclap{\scriptsize\text{$j$-th column}}\phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} \end{array} & & \\ \left(\vphantom{\begin{array}{c} \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \end{array}}\right. & \begin{array}{ccccccccccc} \mathclap{1}\phantom{\ddots} & & & & & & & & & & \\ & \mathclap{\ddots}\phantom{\ddots} & & & & & & & & & \\ & & \mathclap{1}\phantom{\ddots} & & & & & & & & \\ & & & \mathclap{0}\phantom{\ddots} & & & & \mathclap{1}\phantom{\ddots} & & & \\ & & & & \mathclap{1}\phantom{\ddots} & & & & & & \\ & & & & & \mathclap{\ddots}\phantom{\ddots} & & & & & \\ & & & & & & \mathclap{1}\phantom{\ddots} & & & & \\ & & & \mathclap{1}\phantom{\ddots} & & & & \mathclap{0}\phantom{\ddots} & & & \\ & & & & & & & & \mathclap{1}\phantom{\ddots} & & \\ & & & & & & & & & \mathclap{\ddots}\phantom{\ddots} & \\ & & & & & & & & & & \mathclap{1}\phantom{\ddots} \end{array} & \left.\vphantom{\begin{array}{c} \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \end{array}}\right) & \begin{array}{l} \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots}\scriptsize\text{$i$-th row} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots}\scriptsize\text{$j$-th row} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \end{array} \end{array}

Remark: For permutation matrices we have: PijT=Pij−1=PijP_{ij}^T = P_{ij}^{-1} = P_{ij}.

(ii) To multiply row ii by λ≠0\lambda \neq 0, multiply from the left: Si(λ)AS_i(\lambda)A

Si(λ)=⋱⋱⋱i-th column⋱⋱⋱⋱(⋱⋱⋱⋱⋱⋱⋱1⋱⋱⋱1⋱λ⋱1⋱⋱⋱1⋱⋱⋱⋱⋱⋱⋱⋱)⋱⋱⋱⋱i-th row⋱⋱⋱S_i(\lambda) = \begin{array}{cccl} & \begin{array}{ccccccc} \phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \mathclap{\scriptsize\text{$i$-th column}}\phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} \end{array} & & \\ \left(\vphantom{\begin{array}{c} \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \end{array}}\right. & \begin{array}{ccccccc} \mathclap{1}\phantom{\ddots} & & & & & & \\ & \mathclap{\ddots}\phantom{\ddots} & & & & & \\ & & \mathclap{1}\phantom{\ddots} & & & & \\ & & & \mathclap{\lambda}\phantom{\ddots} & & & \\ & & & & \mathclap{1}\phantom{\ddots} & & \\ & & & & & \mathclap{\ddots}\phantom{\ddots} & \\ & & & & & & \mathclap{1}\phantom{\ddots} \end{array} & \left.\vphantom{\begin{array}{c} \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \end{array}}\right) & \begin{array}{l} \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots}\scriptsize\text{$i$-th row} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \end{array} \end{array}

(iii) To replace row ii by (row i+λ⋅i+\lambda \cdot row jj), multiply from the left: Eij(λ)AE_{ij}(\lambda)A

Eij(λ)=⋱⋱⋱i-th column⋱⋱⋱⋱j-th column⋱⋱⋱⋱(⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱1⋱⋱⋱1⋱1⋱λ⋱1⋱⋱⋱1⋱0⋱1⋱1⋱⋱⋱1⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱⋱)⋱⋱⋱⋱i-th row⋱⋱⋱⋱j-th row⋱⋱⋱E_{ij}(\lambda) = \begin{array}{cccl} & \begin{array}{ccccccccccc} \phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \mathclap{\scriptsize\text{$i$-th column}}\phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \mathclap{\scriptsize\text{$j$-th column}}\phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} & \phantom{\ddots} \end{array} & & \\ \left(\vphantom{\begin{array}{c} \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \end{array}}\right. & \begin{array}{ccccccccccc} \mathclap{1}\phantom{\ddots} & & & & & & & & & & \\ & \mathclap{\ddots}\phantom{\ddots} & & & & & & & & & \\ & & \mathclap{1}\phantom{\ddots} & & & & & & & & \\ & & & \mathclap{1}\phantom{\ddots} & & & & \mathclap{\lambda}\phantom{\ddots} & & & \\ & & & & \mathclap{1}\phantom{\ddots} & & & & & & \\ & & & & & \mathclap{\ddots}\phantom{\ddots} & & & & & \\ & & & & & & \mathclap{1}\phantom{\ddots} & & & & \\ & & & \mathclap{0}\phantom{\ddots} & & & & \mathclap{1}\phantom{\ddots} & & & \\ & & & & & & & & \mathclap{1}\phantom{\ddots} & & \\ & & & & & & & & & \mathclap{\ddots}\phantom{\ddots} & \\ & & & & & & & & & & \mathclap{1}\phantom{\ddots} \end{array} & \left.\vphantom{\begin{array}{c} \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \\ \ddots \end{array}}\right) & \begin{array}{l} \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots}\scriptsize\text{$i$-th row} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots}\scriptsize\text{$j$-th row} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \\ \vphantom{\ddots} \end{array} \end{array}
Remark

Above we saw how to perform row operations by multiplying with elementary matrices from the left. If we instead multiply by Pij,Si(λ),Eij(λ)∈Em×mP_{ij}, S_i(\lambda), E_{ij}(\lambda) \in \mathbb{E}^{m \times m} from the right, e.g. APijAP_{ij}, we get column operations.

Algorithm recipe: computing the inverse

Given: A∈En×nA \in \mathbb{E}^{n \times n}
Wanted: A−1A^{-1}

  1. Write down the scheme (A∣𝟙)(A|\text{𝟙})
  2. Use Gaussian elimination to transform AA into the identity matrix
    (A∣𝟙)⇝⋯⇝(𝟙∣A−1)(A|\text{𝟙}) \leadsto \dots \leadsto (\text{𝟙}|A^{-1})
  3. Check A−1A=AA−1=𝟙A^{-1}A = AA^{-1} = \text{𝟙}
Remark

Let A∈En×nA \in \mathbb{E}^{n \times n} be a diagonal matrix with a11,…,ann≠0a_{11}, \dots, a_{nn} \neq 0; then:

A=(a110⋯00⋱⋱⋮⋮⋱⋱00⋯0ann)⟹A−1=(1a110⋯00⋱⋱⋮⋮⋱⋱00⋯01ann).A = \begin{pmatrix} a_{11} & 0 & \cdots & 0\\ 0 & \ddots & \ddots & \vdots\\ \vdots & \ddots & \ddots & 0\\ 0 & \cdots & 0 & a_{nn} \end{pmatrix} \quad \Longrightarrow \quad A^{-1} = \begin{pmatrix} \frac{1}{a_{11}} & 0 & \cdots & 0\\ 0 & \ddots & \ddots & \vdots\\ \vdots & \ddots & \ddots & 0\\ 0 & \cdots & 0 & \frac{1}{a_{nn}} \end{pmatrix}.
Example 1

Given: A∈R3×3A \in \mathbb{R}^{3 \times 3}:

A=(326113−3−2−5)A = \begin{pmatrix} 3 & 2 & 6\\ 1 & 1 & 3\\ -3 & -2 & -5 \end{pmatrix}

Wanted: A−1A^{-1}

Show solutionHide solution (326100113010−3−2−5001)⇝(ii)−l21⋅(i)(3261000131−1310−3−2−5001)\left(\begin{array}{ccc|ccc} 3 & 2 & 6 & 1 & 0 & 0\\ 1 & 1 & 3 & 0 & 1 & 0\\ -3 & -2 & -5 & 0 & 0 & 1 \end{array}\right) \overset{(ii)-l_{21} \cdot (i)}{\leadsto} \left(\begin{array}{ccc|ccc} 3 & 2 & 6 & 1 & 0 & 0\\ 0 & \frac{1}{3} & 1 & -\frac{1}{3} & 1 & 0\\ -3 & -2 & -5 & 0 & 0 & 1 \end{array}\right)⇝(iii)−l31⋅(i)(3261000131−1310001101)⇝(ii)−11⋅(iii)(3261000130−431−1001101)\overset{(iii)-l_{31} \cdot (i)}{\leadsto} \left(\begin{array}{ccc|ccc} 3 & 2 & 6 & 1 & 0 & 0\\ 0 & \frac{1}{3} & 1 & -\frac{1}{3} & 1 & 0\\ 0 & 0 & 1 & 1 & 0 & 1 \end{array}\right) \overset{(ii)- \frac{1}{1} \cdot (iii)}{\leadsto} \left(\begin{array}{ccc|ccc} 3 & 2 & 6 & 1 & 0 & 0\\ 0 & \frac{1}{3} & 0 & -\frac{4}{3} & 1 & -1\\ 0 & 0 & 1 & 1 & 0 & 1 \end{array}\right)⇝(i)−61⋅(iii)(320−50−60130−431−1001101)⇝(i)−21/3⋅(ii)(3003−600130−431−1001101)\overset{(i)- \frac{6}{1} \cdot (iii)}{\leadsto} \left(\begin{array}{ccc|ccc} 3 & 2 & 0 & -5 & 0 & -6\\ 0 & \frac{1}{3} & 0 & -\frac{4}{3} & 1 & -1\\ 0 & 0 & 1 & 1 & 0 & 1 \end{array}\right) \overset{(i)- \frac{2}{1/3} \cdot (ii)}{\leadsto} \left(\begin{array}{ccc|ccc} 3 & 0 & 0 & 3 & -6 & 0\\ 0 & \frac{1}{3} & 0 & -\frac{4}{3} & 1 & -1\\ 0 & 0 & 1 & 1 & 0 & 1 \end{array}\right)⇝(i)⋅13(1001−200130−431−1001101)⇝(ii)⋅3(1001−20010−43−3001101)\overset{(i) \cdot \frac{1}{3}}{\leadsto} \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & 1 & -2 & 0\\ 0 & \frac{1}{3} & 0 & -\frac{4}{3} & 1 & -1\\ 0 & 0 & 1 & 1 & 0 & 1 \end{array}\right) \overset{(ii) \cdot 3}{\leadsto} \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & 1 & -2 & 0\\ 0 & 1 & 0 & -4 & 3 & -3\\ 0 & 0 & 1 & 1 & 0 & 1 \end{array}\right)⟹A−1=(1−20−43−3101)\Longrightarrow \quad A^{-1} = \begin{pmatrix} 1 & -2 & 0\\ -4 & 3 & -3\\ 1 & 0 & 1 \end{pmatrix}

Orthogonal and Unitary Matrices

Definition

A complex n×nn \times n matrix AA is called unitary if AHA=AAH=𝟙A^HA = AA^H = \text{𝟙}.
A real n×nn \times n matrix AA is called orthogonal if ATA=AAT=𝟙A^TA = AA^T = \text{𝟙}.

Satz

If A,B∈En×nA,B \in \mathbb{E}^{n \times n} are unitary (or orthogonal) matrices, then:

(i) AA is regular
(ii) A−1=AHA^{-1} = A^H (or A−1=ATA^{-1} = A^T)
(iii) A−1A^{-1} is unitary (orthogonal)
(iv) ABAB is unitary (orthogonal)

Definition

The Kronecker delta is defined by:

δij={1,i=j0,i≠j\delta_{ij} = \begin{cases} 1, \quad i=j\\ 0, \quad i \neq j \end{cases}
Definition unit vectors
e1:=(10⋮0),e2:=(010⋮0),ei:=(0⋮010⋮0),en:=(0⋮01)e_1 := \begin{pmatrix} 1\\ 0\\ \vdots\\ 0 \end{pmatrix}, e_2 := \begin{pmatrix} 0\\ 1\\ 0\\ \vdots\\ 0 \end{pmatrix}, e_i := \begin{pmatrix} 0\\ \vdots\\ 0\\ 1\\ 0\\ \vdots\\ 0 \end{pmatrix}, e_n := \begin{pmatrix} 0\\ \vdots\\ 0\\ 1 \end{pmatrix}
Example 2
  • ⟨e1,e2⟩=0\langle e_1, e_2 \rangle = 0
  • ⟨e1,e1⟩=1\langle e_1, e_1 \rangle = 1
Definition orthonormal

Let a,b∈Ena,b \in \mathbb{E}^n. The vectors a,ba,b are orthonormal if the following conditions are satisfied:

  1. The vectors are normalised, i.e.: ∥a∥=1and∥b∥=1.\Vert a \Vert = 1 \quad \text{and} \quad \Vert b \Vert = 1.
  2. The vectors are orthogonal, i.e.: ⟨a,b⟩=0.\langle a, b \rangle = 0.
Remark

For an orthogonal matrix A∈En×nA \in \mathbb{E}^{n \times n} of the form A=(a1∣...∣an)A = (a_1 | ... | a_n), the column vectors a1,...,ana_1, ..., a_n are pairwise orthonormal.