|
Hooman Fatoorehchi Assistant Professor School of Chemical Engineering University of Tehran, Tehran, Iran  Some Mathematical Theorems (presented unorderly) Theorem: $(\textbf{AB})^T = \textbf{B}^T \textbf{A}^T$, where superscript $T$ denotes matrix transpose. Theorem: $eig(\textbf{A})=eig(\textbf{A}^T)$. Also, $\textbf{A}$ and $\textbf{A}^T$ do not commute necessarily. Theorem: Let $\textbf{A},\textbf{B} \in \mathbb{R}^{n \times n}$. Also, let $\textbf{B}$ be symmetric. It holds that $\textbf{A}\textbf{B}\textbf{A}^T$ and $\textbf{A}^T\textbf{B}\textbf{A}$ are symmetric (note that $\textbf{A}\textbf{B}\textbf{A}^T \ne \textbf{A}^T\textbf{B}\textbf{A}$). Proof: $(\textbf{A}\textbf{B}\textbf{A}^T)^T= (\textbf{A}^T)^T (\textbf{A}\textbf{B})^T=\textbf{A}\textbf{B}^T\textbf{A}^T=\textbf{A}\textbf{B}\textbf{A}^T$. Theorem: Trace is similarity-invariant, i.e., $\text{trace}(\textbf{A})=\text{trace}(\textbf{S}\textbf{A}\textbf{S}^{-1})$. Theorem: For square matrices $\textbf{A}$ and $\textbf{B}$, we have $det(\textbf{AB})=det(\textbf{A})\hspace{0.05cm} det(\textbf{B})$. Corollary: By knowing $\textbf{A}\textbf{A}^{-1}=\textbf{I}$, we conclude that $det\left( \textbf{A}^{-1}\right)=\cfrac{1}{det\left( \textbf{A} \right)}$. Theorem: Similar matrices have the same determinant. Proof: $\textbf{A} \sim \textbf{B}$ imposes by definition that there exists an invertible square matrix $\textbf{S}$ such that $\textbf{B}=\textbf{S}^{-1}\textbf{A}\textbf{S}$. Therefore, $det (\textbf{B})=det(\textbf{S}^{-1})\hspace{0.05cm}det(\textbf{A})\hspace{0.05cm}det(\textbf{S})=det(\textbf{A}).$ Theorem: $\textbf{A} \sim \textbf{B} \rightarrow p(\textbf{A})\sim p(\textbf{B})$, where $\sim$ denotes matrix similarity and $p$ is a real (also can be complex) polynomial. Theorem: Swapping the rows or columns of a matrix changes only the sign of its determinant; but totally changes its eigenvalues. Theorem: If we swap the rows and columns of a matrix in an identical pattern, a similar matrix is obtained (in other words, the eigenvalues and the determinant are preserved). For instance, swap column 1 and column 4 and then, swap row 1 and row 4. Theorem: If $\textbf{A}$ and $\textbf{B}$ are positive definite matrices, then $\textbf{A+B}$ is positive definite. Theorem: If $\textbf{A}$ and $\textbf{B}$ are positive definite and also they commute, then $\textbf{AB}$ is positive definite. Theorem: $det \left( \begin{bmatrix} \mathbf{A} & \mathbf{B} \\ \mathbf{B} & \mathbf{A} \end{bmatrix} \right)=det \left( \mathbf{A-B}\right)det \left( \mathbf{A+B}\right)$. Theorem: A strictly diagonally dominant (SDD) matrix is invertible. Theorem: As a lower-bound for the norm of an inverse matrix, it holds $\Vert \textbf{A}^{-1} \Vert \geq \Vert \textbf{A} \Vert^{-1} $. Theorem: As an upper-bound for the norm of an inverse matrix, it holds $\Vert \textbf{A}^{-1} \Vert \leq $ $\Large \frac{\Vert \textbf{A} \Vert^{n-1}}{|det(\textbf{A})|} $. Note that matrix $\textbf{A}$ is n-by-n. Theorem: Eigenvalues of a Hermitian matrix are real. Corollary: The eigenvalues of a real symmetric matrix are real. Theorem: Hadamard's inequality (an upper-bound for the determinant of a matrix) $|det(\textbf{A})| \leq \displaystyle \prod_{i=1}^{n} \sqrt{\sum\nolimits_{j=1}^{n} |a_{ij}|^2}$. Theorem: $ \forall n \in \mathbb{N}$, it holds that $n^n \geq n!$. Proof by induction. The equality is only for $n=1$. Theorem: If $p(x)$ is a polynomial, then $\textbf{B}=p(\textbf{A})$ commutes with $\textbf{A}$, i.e. $\textbf{AB}=\textbf{BA}$. Additionally, a matrix commutes with its inverse (if it has one), as well as with its matrix exponential. Theorem: If matrix $\textbf{AB}$ is invertible, then matrices $\textbf{A}$ and $\textbf{B}$ are invertible. Theorem: Assume that matrix $\textbf{A}$ is invertible. Then, matrix $\textbf{B}$ is invertible iff $\textbf{AB}$ is invertible. Also, $\textbf{B}$ is invertible iff $\textbf{BA}$ is invertible. Theorem: Row operations and column operations both preserve invertibility. The last three theorems are useful for proving the invertibility of a given matrix. Theorem: Let square matrices $\textbf{A}$ and $\textbf{B}$ be diagonalizable. Then, $\textbf{A}$ and $\textbf{B}$ commute if and only if they are simultaneously diagonalizable. Proof: Horn RA, Johnson CR, Matrix Analysis. Cambridge University Press, 2012. Corollary: If square matrices $\textbf{A}$ and $\textbf{B}$ are diagonalizable and they commute, then they are simultaneously diagonalizable. Theorem: If square matrices $\textbf{A}$ and $\textbf{B}$ commute, i.e. $\textbf{AB}=\textbf{BA}$, then $eig \left( \textbf{AB}\right)=eig\left(\textbf{A}\right)eig\left(\textbf{B}\right)$. Theorem: If square matrices $\textbf{A}$ and $\textbf{B}$ commute, i.e. $\textbf{AB}=\textbf{BA}$, then $eig \left( \textbf{A+B}\right)=eig\left(\textbf{A}\right)+eig\left(\textbf{B}\right)$. Theorem: If a matrix has distinct eigenvalues, then it is diagonalizable. Theorem: A square matrix is diagonalizable if it is similar to a diagonalizable matrix. Theorem: Matrix $\textbf{A}$ is negative definite if and only if $-\textbf{A}$ is positive definite. Theorem: Let matrix $\textbf{Y}$ satisfy the equation $\textbf{A}^T \textbf{Y} +\textbf{Y}\textbf{A}=-\textbf{I}$. The real matrix $\textbf{A}$ is Hurwitz stable if and only if $\textbf{Y}$ is positive definite. Proof: Richard Bellman, Introduction to Matrix Analysis. 2nd Ed., SIAM, Philadelphia, 1997. Theorem: Let $\textbf{A} \in \mathbb{C}^{n \times n}$ having real eigenvalues $\lambda_i \in \mathbb{R}, \hspace{0.25 cm} 1\le i \le n$ and being ordered as $\lambda_1 \ge \lambda_2 \ge \cdots \ge \lambda_n$. For $1 \le k \le n$, it holds that $\lambda_1+\cdots +\lambda_k \ge k \left[ \cfrac{\text{trace}(\textbf{A})}{n} + \sqrt{\cfrac{1}{n(n-1)}\left(\text{trace}(\textbf{A}^2)-\cfrac{\left( \text{trace}(\textbf{A}) \right)^2}{n} \right) } \hspace{0.15cm}\right]$; $k\le \cfrac{n}{2}$ $\lambda_{n-k+1}+\cdots +\lambda_n \le k \left[ \cfrac{\text{trace}(\textbf{A})}{n} - \sqrt{\cfrac{1}{n(n-1)}\left(\text{trace}(\textbf{A}^2)-\cfrac{\left( \text{trace}(\textbf{A}) \right)^2}{n} \right) } \hspace{0.15cm}\right]$; $k\le \cfrac{n}{2}$ $\lambda_1+\cdots +\lambda_k \ge k \hspace{0.1cm} \cfrac{\text{trace}(\textbf{A})}{n} +(n-k) \sqrt{\cfrac{1}{n(n-1)}\left(\text{trace}(\textbf{A}^2)-\cfrac{\left( \text{trace}(\textbf{A}) \right)^2}{n} \right) }$; $k\ge \cfrac{n}{2}$ $\lambda_{n-k+1}+\cdots +\lambda_n \le k \hspace{0.1cm} \cfrac{\text{trace}(\textbf{A})}{n} -(n-k) \sqrt{\cfrac{1}{n(n-1)}\left(\text{trace}(\textbf{A}^2)-\cfrac{\left( \text{trace}(\textbf{A}) \right)^2}{n} \right) }$; $k\ge \cfrac{n}{2}$ Proof: See Merikoski and Virtanen 1997 (DOI: 10.1016/S0024-3795(97)00067-0) or originally, Wolkowicz and Styan 1980 (DOI: 10.1016/0024-3795(80)90258-X). Theorem: Let $\textbf{A} \in \mathbb{C}^{n\times n}$ and define $\textbf{B}=\cfrac{1}{2} \left( \textbf{A}+\textbf{A}^*\right)$ as well as $\textbf{C}=\cfrac{1}{2i} \left( \textbf{A}-\textbf{A}^*\right)$, where superscript asterisk denotes conjugate transpose and $i=\sqrt{-1}$. Also, let $\Theta=\sqrt{ \cfrac{\text{trace}\left( \textbf{B}^2 \right)}{n}-\left( \cfrac{\text{Re}\left( \text{trace}\left( \textbf{A} \right) \right)}{n} \right)^2\hspace{0.1cm} }$ and $\Omega=\sqrt{ \cfrac{\text{trace}\left( \textbf{C}^2 \right)}{n}-\left( \cfrac{\text{Im}\left( \text{trace}\left( \textbf{A} \right) \right)}{n} \right)^2\hspace{0.1cm} }$. It holds that $\cfrac{\text{Re}\left( \text{trace}\left( \textbf{A} \right) \right)}{n}-\Theta\hspace{0.1cm}\sqrt{\left( n-1 \right)\hspace{0.1cm}}\le \text{Re}\left( \lambda_j \right) \le \cfrac{\text{Re}\left( \text{trace}\left( \textbf{A} \right) \right)}{n}+\Theta\hspace{0.1cm}\sqrt{\left( n-1 \right)\hspace{0.1cm}}$ $\cfrac{\text{Im}\left( \text{trace}\left( \textbf{A} \right) \right)}{n}-\Omega\hspace{0.1cm}\sqrt{\left( n-1 \right)\hspace{0.1cm}}\le \text{Im}\left( \lambda_j \right) \le \cfrac{\text{Im}\left( \text{trace}\left( \textbf{A} \right) \right)}{n}+\Omega\hspace{0.1cm}\sqrt{\left( n-1 \right)\hspace{0.1cm}}$ where $\lambda_j$ denotes the eigenvalues of $\textbf{A}$. Proof: Wolkowicz and Styan 1980 (DOI: 10.1016/0024-3795(80)90258-X). Theorem: Let $p(x)=a_0 x^n + a_1 x^{n-1} +\cdots + a_n$. Also, let $q(x)$ be another polynomial with coefficients arranged in the reverse order of those in $p(x)$, i.e., $q(x)=a_n x^n + a_{n-1}x^{n-1}+\cdots+a_0$. It holds that the roots of $p(x)$ are the reciprocals of the roots of $q(x)$. Theorem: Let $\textbf{A} \in \mathbb{C}^{n \times n}$ and nonsingular. Then, it holds that $eig(\textbf{A}^{-1})=\cfrac{1}{eig(\textbf{A})}$. Proof: $\textbf{A}\textbf{v}=eig(\textbf{A})\textbf{v} \rightarrow \textbf{A}^{-1}\textbf{A}\textbf{v}=eig(A)\textbf{A}^{-1}\textbf{v}\rightarrow \textbf{A}^{-1}\textbf{v}=\cfrac{1}{eig(\textbf{A})}\textbf{v}.$ My Observations During Different Works Remark: Norms of similar matrices can be different. As an example, consider, $\textbf{A}= \begin{bmatrix} 1 & 2 & 3\\4 & 0.5 & 6\\7 & 8 & 0.9 \end{bmatrix}, \textbf{D}= \begin{bmatrix} 11.8 & 0 & 0\\0 & 10.5 & 0\\0 & 0 & 10.3 \end{bmatrix} , \textbf{B}=\textbf{DAD$^{-1}$}= \begin{bmatrix} 1 & 2.2476 & 3.4369\\3.5593 & 0.5 & 6.1165\\6.1102 & 7.8476 & 0.9 \end{bmatrix}$. So, $\textbf{A} \sim \textbf{B}$, i.e. $eig(\textbf{A})=eig(\textbf{B})$. $\|\textbf{A}\|_{\infty} = 15.9, \|\textbf{B}\|_{\infty}=14.8578$. Methods and Techniques The General Solution of First-Order Linear ODEs $\frac{dy}{dx}+p(x) y = g(x) \rightarrow y(x) = \frac{1}{\exp\left(\int^x p(t)dt\right)} \left[ \int^x g(\xi)\exp(\int^{\xi}p(t)dt )d\xi +C \right]$ The Laplace Expansion Formula for Determinant Calculation of a Matrix $det(\textbf{A})=\sum_{i=1}^{n} a_{ij}c_{ij}=\sum_{i=1}^{n} a_{ij}\hspace{0.1cm}(-1)^{i+j}M_{ij}$, $j=1,2,\dots, n$ $det(\textbf{A})=\sum_{j=1}^{n} a_{ij}c_{ij}=\sum_{j=1}^{n} a_{ij}\hspace{0.1cm}(-1)^{i+j}M_{ij}$, $i=1,2,\dots, n$ where $c_{ij}$ is called the $ij$ cofactor of matrix $\textbf{A}$ and $M_{ij}$ is the $ij$ minor of matrix $\textbf{A}$. The $ij$ minor of matrix $\textbf{A}$ is the determinant of the matrix (one size smaller) obtained by deleting the $i$th row and the $j$th column of matrix $\textbf{A}$. General Algorithm for Inversion of a Matrix $\textbf{A}^{-1}=\cfrac{1}{det(\textbf{A})} adj(\textbf{A})$, where $adj(\textbf{A})$ is called the adjugate or adjoint matrix of $\textbf{A}$. By definition, it is the transpose of the cofactors matrix. $adj(\textbf{A})= \begin{bmatrix} c_{11} & c_{12} & \cdots & c_{1n} \\ c_{21} & c_{22} & \cdots & c_{2n} \\ \vdots & \vdots & \cdots & \vdots \\c_{n1} & c_{n2} & \cdots & c_{nn} \end{bmatrix}^T=\textbf{C}^T $, Example: $\textbf{A}= \begin{bmatrix} 1 & 2 & 3 \\ 0 & -1 & -2 \\ 4 & 5 & 1 \end{bmatrix}$. $det(\textbf{A})= 0\times M_{21} \times(-1)^{2+1}+(-1)\times M_{22}\times (-1)^{2+2}+(-2)\times M_{23} \times (-1)^{2+3}=-\hspace{0.05cm}det \left( \begin{bmatrix} 1 & 3 \\ 4 & 1 \end{bmatrix} \right)+ 2 \hspace{0.1cm}det \left( \begin{bmatrix} 1 & 2 \\ 4 & 5 \end{bmatrix} \right)=11+2\times(-3)=5$. $c_{11}=(-1)^{1+1}\hspace{0.05cm} det \left( \begin{bmatrix} -1 & -2 \\ 5 & 1 \end{bmatrix} \right)=9$, $c_{12}=(-1)^{1+2}\hspace{0.05cm} det \left( \begin{bmatrix} 0 & -2 \\ 4 & 1 \end{bmatrix} \right)=-8$, $c_{13}=(-1)^{1+3}\hspace{0.05cm} det \left( \begin{bmatrix} 0 & -1 \\ 4 & 5 \end{bmatrix} \right)=4$, $c_{21}=(-1)^{2+1}\hspace{0.05cm} det \left( \begin{bmatrix} 2 & 3 \\ 5 & 1 \end{bmatrix} \right)=13$, $c_{22}=(-1)^{2+2}\hspace{0.05cm} det \left( \begin{bmatrix} 1 & 3 \\ 4 & 1 \end{bmatrix} \right)=-11$, $c_{23}=(-1)^{2+3}\hspace{0.05cm} det \left( \begin{bmatrix} 1 & 2 \\ 4 & 5 \end{bmatrix} \right)=3$, $c_{31}=(-1)^{3+1}\hspace{0.05cm} det \left( \begin{bmatrix} 2 & 3 \\ -1 & -2 \end{bmatrix} \right)=-1$, $c_{32}=(-1)^{3+2}\hspace{0.05cm} det \left( \begin{bmatrix} 1 & 3 \\ 0 & -2 \end{bmatrix} \right)=2$, $c_{33}=(-1)^{3+3}\hspace{0.05cm} det \left( \begin{bmatrix} 1 & 2 \\ 0 & -1 \end{bmatrix} \right)=-1$. $adj(\textbf{A})= \begin{bmatrix} 9 & -8 & 4 \\ 13 & -11 & 3 \\ -1 & 2 & -1 \end{bmatrix}^T= \begin{bmatrix} 9 & 13 & -1 \\ -8 & -11 & 2 \\ 4 & 3 & -1 \end{bmatrix}$. $\textbf{A}^{-1}=\frac{1}{5} \begin{bmatrix} 9 & 13 & -1 \\ -8 & -11 & 2 \\ 4 & 3 & -1 \end{bmatrix}=\begin{bmatrix} 1.8 & 2.6 & -0.2 \\ -1.6 & -2.2 & 0.4 \\ 0.8 & 0.6 & -0.2 \end{bmatrix}$. The Faddeev-Leverrier Algorithm for Characteristic Polynomial of Matrices and Matrix Inversion Let $\phi(x)=\sum_{i=0}^{n} a_i x^{n-i}$ be the characteristic polynomial of matrix $\textbf{A}$. The following recursive relation gives the coefficients $a_i$: $\begin{cases} \textbf{T}_1=\textbf{A}, a_0=1 & \\ a_i=-\frac{1}{i}\text{trace}(\textbf{T$_i$}), & 1 \le i \le n \\ \textbf{T}_{i+1}=\textbf{A}(\textbf{T}_i+a_i\textbf{I}), & 1 \le i < n \end{cases}$ It is clear that trace{} is the sum of the elements on the main diagonal of its input matrix and $\textbf{I}$ is the identity matrix. Also, $\textbf{A}^{-1}=-\cfrac{1}{a_n} \left( \textbf{T}_{n-1}+a_{n-1}\textbf{I} \right)$. Integrals $\int_0^{\infty} \exp \left(-px\right)\sin\left(qx+r\right)dx= \cfrac{p\sin(r)+q\cos(r)}{p^2+q^2}$ $\int_0^{\infty} \exp \left(-px\right)\cos\left(qx+r\right)dx= \cfrac{p\cos(r)-q\sin(r)}{p^2+q^2}$ $\int_0^{\infty} \frac{\sin\left(\alpha x \right)}{x} dx = \frac{\pi}{2}\text{sgn}\left(\alpha \right) $ (Dirichlet integral) $\int_a^b f(x) dx = \int_a^b f(a+b-x) dx $ (The King property) $\int_{-\infty}^{+\infty} \frac{1}{x^2+1} dx = \pi $ $\int_{0}^{a} f(x) dx = \int_{0}^{a} f(a-x) dx $ $f(2a-x)=f(x)\leftrightarrow \int_{0}^{2a} f(x) dx = 2\int_{0}^{a} f(x) dx $ $ \int \sqrt{a^2+x^2} dx = \frac{x}{2} \sqrt{a^2+x^2}+\frac{a^2}{2}\ln {\left(x +\sqrt{a^2+x^2} \hspace{0.07cm}\right)}+C $ $ \int \frac{dx}{\sqrt{a^2+x^2}} = \ln{\left(x+\sqrt{a^2+x^2} \hspace{0.07cm}\right)}+C $ $\int_{0}^{1}x^{a-1}(1-x)^{b-1}dx=\frac{\Gamma(a)\hspace{0.05cm}\Gamma(b)}{\Gamma(a+b)} $ Theorem (Lobachevsky integral formula): Let $f(x)$ be continuous and satisfy the $\pi-$periodic assumption $f(x+\pi)=f(x)$, and $f(\pi-x)=f(x)$, for $0 \le x < \infty$; Then, $\int_0^{\infty} \frac{\sin^2 x}{x^2} f(x) dx = \int_0^{\infty} \frac{\sin x}{x} f(x)dx = \int_0^{\frac{\pi}{2}}f(x)dx$. For example, if we take $f(x)=1$, then we will have $\int_0^{\infty} \frac{\sin^2 x}{x^2} dx = \int_0^{\infty} \frac{\sin x}{x} dx = \frac{\pi}{2}$. Limits and Series $ n! \sim \sqrt{2\pi n} \left(\frac{n}{e} \right)^n$ (Stirling's asymptotic formula) $\lim\limits_{n\to +\infty} \frac{n! \hspace{0.25 cm} e^{n} }{\sqrt{2\pi} \hspace{0.25 cm} n^{n+\frac{1}{2}} }=1$ Identities $ p\cos (x)+q\sin (x) = r\sin (x+\theta)$ where $r=\sqrt{p^2+q^2}; \hspace{0.7 cm} \tan (\theta)=\frac{p}{q}$ |