Practice Problems 4

Lecture 4: Matrix Multiplication, Vectors, and Special Matrices

Problem numbers refer to the textbook (Chiang & Wainwright, 4th edition). These are not graded, but you should attempt them all before the quiz. Try each part yourself first, then expand the Solution to check.

Exercise 4.4

5. (e) Find (i) $C=AB$, and (ii) $D=BA$, if $$A=\left[\begin{array}{c}-2\\ 4\\ 7\end{array}\right]B=\left[\begin{array}{ccc}3& 6& -2\end{array}\right]$$

Solution

$$C=AB={\left[\begin{array}{ccc}-6& -12& 4\\ 12& 24& -8\\ 21& 42& -14\end{array}\right]}_{3\times 3}D=BA=[3\times (-2)+6\times 4+(-2)\times 7]=[4{]}_{1\times 1}$$

7. If the matrix $A$ in Example 5 had all its four elements nonzero, would $x^{T}Ax$ still give a weighted sum of squares? Would the associative law still apply?

Solution

In Example 5, $x=\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]$ and $A=\left[\begin{array}{cc}{a}_{11}& 0\\ 0& a_{22}\end{array}\right]$, in which case $$x^{T}Ax=\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]\left[\begin{array}{cc}{a}_{11}& 0\\ 0& a_{22}\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=\underset{\text{weighted sum of squares}}{\underbrace{a_{11}{x}_1^2+a_{22}{x}_2^2}}=\sum _{i=1}^2{a}_{ii}{x}_i^2$$

But if all four elements of $A$ are nonzero, $$\begin{array}{rl}{x}^{T}Ax& =\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]\left[\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]\\ & =a_{11}{x}_1^2+(a_{21}+a_{12})x_1{x}_2+a_{22}{x}_2^2\end{array}$$ which is no longer a weighted sum of squares. The associative law $\left(x^{T}A\right)x=x^T(Ax)$ applies in both cases (after all, it’s a law!) as all the products are possible.

Exercise 4.5

1. Given $$A=\left[\begin{array}{ccc}-1& 5& 7\\ 0& -2& 4\end{array}\right],b=\left[\begin{array}{c}9\\ 6\\ 0\end{array}\right],x=\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right],$$ calculate the following, indicating the dimension of the identity matrix used in each case.

1. $AI$

Solution

$AI=\left[\begin{array}{ccc}-1& 5& 7\\ 0& -2& 4\end{array}\right]{\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]}_{3\times 3}=\left[\begin{array}{ccc}-1& 5& 7\\ 0& -2& 4\end{array}\right]=A$

2. $IA$

Solution

$IA={\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]}_{2\times 2}\left[\begin{array}{ccc}-1& 5& 7\\ 0& -2& 4\end{array}\right]=\left[\begin{array}{ccc}-1& 5& 7\\ 0& -2& 4\end{array}\right]=A$

3. $Ix$

Solution

$Ix={\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]}_{2\times 2}\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=x$

4. $x^{T}I$

Solution

$x^{T}I=\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]{\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]}_{2\times 2}=\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]=x^T$

4. Show that the diagonal matrix $$\left[\begin{array}{cccc}{a}_{11}& 0& \cdots & 0\\ 0& a_{22}& \cdots & 0\\ \cdots & \cdots & \cdots & \cdots \\ 0& 0& \cdots & a_{nn}\end{array}\right]$$ can be idempotent only if each diagonal element is either 1 or 0. How many different numerical idempotent diagonal matrices of dimension $n\times n$ can be constructed altogether from such a matrix?

Solution

Start with a $2\times 2$ diagonal matrix: $$\left[\begin{array}{cc}{a}_{11}& 0\\ 0& a_{22}\end{array}\right]\left[\begin{array}{cc}{a}_{11}& 0\\ 0& a_{22}\end{array}\right]=\left[\begin{array}{cc}{a}_{11}^2& 0\\ 0& a_{22}^2\end{array}\right]$$

$x=x^2$ only for $x=0,1$, so $a_{11}$ and $a_{22}$ can each be 0 or 1. The $2\times 2$ idempotent diagonal matrices are $$\left[\begin{array}{cc}1& 0\\ 0& 0\end{array}\right],\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],\left[\begin{array}{cc}0& 0\\ 0& 0\end{array}\right],\left[\begin{array}{cc}0& 0\\ 0& 1\end{array}\right]$$

More generally, an $n\times n$ matrix allows $2^n$ such matrices: there are $n$ diagonal elements, each of which can take two values.

Exercise 4.6

2. Given $A=\left[\begin{array}{cc}0& 4\\ -1& 3\end{array}\right],B=\left[\begin{array}{cc}3& -8\\ 0& 1\end{array}\right]$, and $C=\left[\begin{array}{ccc}1& 0& 9\\ 6& 1& 1\end{array}\right]$, verify that

1. $(A+B{)}^T=A^T+B^T$

Solution

$$A+B=\left[\begin{array}{cc}3& -4\\ -1& 4\end{array}\right],\text{ so }(A+B{)}^T=\left[\begin{array}{cc}3& -1\\ -4& 4\end{array}\right]$$ $$A^T+B^T=\left[\begin{array}{cc}0& -1\\ 4& 3\end{array}\right]+\left[\begin{array}{cc}3& 0\\ -8& 1\end{array}\right]=\left[\begin{array}{cc}3& -1\\ -4& 4\end{array}\right]✓$$

2. $(AC{)}^T=C^T{A}^T$

Solution

$$AC={\left[\begin{array}{cc}0& 4\\ -1& 3\end{array}\right]}_{2\times 2}{\left[\begin{array}{ccc}1& 0& 9\\ 6& 1& 1\end{array}\right]}_{2\times 3}={\left[\begin{array}{ccc}24& 4& 4\\ 17& 3& -6\end{array}\right]}_{2\times 3}$$ $$C^T{A}^T={\left[\begin{array}{cc}1& 6\\ 0& 1\\ 9& 1\end{array}\right]}_{3\times 2}{\left[\begin{array}{cc}0& -1\\ 4& 3\end{array}\right]}_{2\times 2}={\left[\begin{array}{cc}24& 17\\ 4& 3\\ 4& -6\end{array}\right]}_{3\times 2}=(AC{)}^T✓$$

6. Let $A=I-X{\left(X^{T}X\right)}^{-1}{X}^T$.

1. Must $A$ be square? Must $\left(X^{T}X\right)$ be square? Must $X$ be square?

Solution

Say the dimension of $X$ is $m\times n$. Then the dimension of $X_{n\times m}^T{X}_{m\times n}$ is $n\times n$, so the dimension of ${\left(X^{T}X\right)}^{-1}$ is also $n\times n$. This implies that the dimension of $X_{m\times n}{\left(X^{T}X\right)}_{n\times n}^{-1}{X}_{n\times m}^T$ is $m\times m$. Hence, $X^{T}X$ and $A$ must be square matrices, but $X$ need not be square.

2. Show that matrix $A$ is idempotent.

Solution

To prove a matrix is idempotent, we need to show $AA=A$: $$\begin{array}{rl}AA& =(I-X{\left(X^{T}X\right)}^{-1}{X}^T)(I-X{\left(X^{T}X\right)}^{-1}{X}^T)\\ & =I-X{\left(X^{T}X\right)}^{-1}{X}^T-X{\left(X^{T}X\right)}^{-1}{X}^T+X\underset{I}{\underbrace{{\left(X^{T}X\right)}^{-1}{X}^{T}X}}{\left(X^{T}X\right)}^{-1}{X}^T\\ & =I-X{\left(X^{T}X\right)}^{-1}{X}^T=A\end{array}$$

Exercise 5.1

3. Are the rows linearly independent in each of the following?

1. $\left[\begin{array}{cc}24& 8\\ 9& -3\end{array}\right]$
2. $\left[\begin{array}{cc}2& 0\\ 0& 2\end{array}\right]$
3. $\left[\begin{array}{cc}0& 4\\ 3& 2\end{array}\right]$
4. $\left[\begin{array}{cc}-1& 5\\ 2& -10\end{array}\right]$

Solution

1. Yes
2. Yes
3. Yes
4. No; the second row $=-2\times$ the first row.

4. Check whether the columns of each matrix in Prob. 3 are also linearly independent. Do you get the same answer as for row independence?

Solution

Yes, we get the same answer.