Mathematics for Data Science: the questions likely to come

a) Prove that if a matrix $A$ is symmetric, then any two distinct eigenvectors corresponding to different eigenvalues are orthogonal. b) Show that the matrix $A = \begin{pmatrix} 2 & -1 \\ 1 & 4 \end{pmatrix}$ is not diagonalizable.

Find an SVD of the matrix $\begin{pmatrix} 1 & 1 \\ 0 & 1 \\ 1 & 0 \end{pmatrix}$.

OR

a) Prove that if $A$ is an $m \times n$ matrix, then all the eigenvalues of $A^T A$ are non-negative. b) Find the eigenvalues and eigenvectors of $A^T A$ where $A = \begin{pmatrix} 1 & 1 \\ 1 & 1 \\ -2 & 1 \end{pmatrix}$.

a) Let $v_1, v_2$ be the eigenvectors associated with the eigenvalues $\lambda_1, \lambda_2$ of a $2 \times 2$ symmetric matrix $A$ respectively. Prove that $A = \lambda_1 v_1 v_1^T + \lambda_2 v_2 v_2^T$.

b) Find an orthogonal matrix $V$ which diagonalizes the matrix $\begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}$. Also check that $V^T A V = \Lambda$, where $\Lambda$ is a diagonal matrix.

OR

a) Prove that if $A \in \mathbf{R}^{3 \times 3}$ with a quadratic form in 3 variables, then there is a symmetric matrix $B \in \mathbf{R}^{3 \times 3}$ such that

b) Classify the quadratic form: $-x_1^2 - 2x_1x_2 - x_2^2$. Then make a change of variable, $x = Py$, that transforms the quadratic form into one with no cross-product term. Write the new quadratic form. Determine $P$.

Let $A$ be an $m \times n$ matrix. Prove that

a) $A^T A$ is a square matrix. b) $A^T A$ is symmetric and so it is orthogonally diagonalizable. c) All the eigenvalues of $A^T A$ are non-negative.

Let $A = \begin{pmatrix} 3 & 2 \\ 2 & 1 \end{pmatrix}$ be a symmetric matrix. Find the orthogonal matrix $D$ such that $D^{-1} A D$ is a diagonal matrix. Let $u_1, u_2, \ldots, u_n$ be the eigenvectors associated with the eigenvalues $\lambda_1, \lambda_2, \ldots, \lambda_n$ of a $n \times n$ symmetric matrix $B$ respectively, then prove that $B = \lambda_1 u_1 u_1^T + \lambda_2 u_2 u_2^T + \ldots + \lambda_n u_n u_n^T$. (3 + 3)

Discuss in brief about singular value decomposition? Find the singular value decomposition of the matrix $\begin{pmatrix} 3 & 1 & 1 \\ -1 & 3 & 1 \end{pmatrix}$. (1 + 5)

Prove that if $B$ is a symmetric bilinear function on $\mathbb{R}^n$, then it is of the form $B = B_A(v, w) = v^T A w$, for some unique symmetric matrix $A$.

OR

Express the quadratic form $Q(x) = x_1 x_2 - x_1 x_3 + x_2 x_3$ as a sum of squares.

Find the SVD of $A = \begin{pmatrix}3 & 2 & 2\\2 & 3 & -2\end{pmatrix}$. If $A$ an invertible $n \times n$ matrix, what is the relationship between the singular values of $A$ and $A^{-1}$?

Find the singular values of the matrix $\begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \end{pmatrix}$.

Let $Q(x) = 7x_1^2 + x_2^2 + 7x_3^2 - 8x_1x_2 - 4x_1x_3 - 8x_2x_3$. Find a unit vector $x$ in $\mathbf{R}^3$ at which $Q(x)$ is maximized, subject to $x^T x = 1$.

Find the singular values of the matrix $\begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 2 \end{pmatrix}$.

Find the maximum value of $Q(x) = x_1^2 + x_2^2 - 10x_1x_2$ subject to the constraints $x^T x = 1$. Find a unit vector $x$ in $\mathbf{R}^2$ at which $Q(x)$ is maximized.

What is quadratic form? Let $S$ be a $3 \times 3$ square matrix with a quadratic form in 3 variables. Then there exists a $3 \times 3$ symmetric matrix $T$ such that $X^T S X = X^T T X, \forall X \in R^3$. (1 + 2)

Without calculation, find one eigenvalue and two linearly independent eigenvectors of

Justify your answer.

Let $Q(x) = 3x_1^2 + 9x_2^2 + 8x_1 x_2$. Find (a) the maximum value of $Q(x)$ subject to the constraint $x^T x = 1$, (b) a unit vector $u$ where this maximum is attained, and (c) the maximum of $Q(x)$ subject to the constraints $x^T x = 1$ and $x^T u = 0$.