> For the complete documentation index, see [llms.txt](https://amanalok.gitbook.io/linear-algebra/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://amanalok.gitbook.io/linear-algebra/eigen-decomposition.md).

# Eigen Decomposition

Eigen Decomposition is extremely useful for square symmetric matrices.

The physical interpretation can be given as: *"Every real matrix can be thought of as a combination of rotation and stretching."*

$$
A\overrightarrow{v} = \overrightarrow{w}
\\
A \in \mathbb{R}^{n \times n}
\hspace{0.5cm}
\overrightarrow{v} \in \mathbb{R}^{n \times 1}
\hspace{0.5cm}
\overrightarrow{w} \in \mathbb{R}^{n \times 1}
$$

### Eigen Vectors

Eigen Vectors of a matrix are those special vectors that only stretch under the action of a matrix.

### Eigen Values

Eigen Values are the factor by which eigen vectors stretch.

$$
Av = \lambda v
$$

### Eigen Decomposition Derivation​

Eigen Decomposition is that set of vectors which only stretch under the action of matrix $$A$$​. Let's say, $$A (\mathbb{R}^{n \times n})$$​ has $$n$$ linearly independent eigenvectors:

$$
\begin{Bmatrix}
v^1, v^2, v^3, ..., v^n\\
\end{Bmatrix}
$$

​Concatenate all the vectors (as columns) and make a eigen vector matrix $$V$$​.

$$
V = \begin{bmatrix}
v^1 & v^2 & ... & v^n\\
\end{bmatrix}
$$

Then, we concatenate the corresponding eigen values into a diagonal matrix:

$$
\Lambda = diag(\lambda\_1, \lambda\_2, ..., \lambda\_n)
$$

Eigen Decomposition ​(Factorisation) of $$A$$​ is given by the following equation:

$$
A = V\Lambda V^{-1}
$$

For real symmetric matrices, we have real eigen vectors and values:

$$
A = Q \Lambda Q^T
\hspace{0.3cm}
where
\hspace{0.1cm}
Q^T = Q^{-1}
$$

Thus, $$Q$$ is orthogonal. This also means $$Q$$​ is a rotation matrix. So, for the following action:

$$
Av = Q \Lambda Q^Tv
$$

​$$Q^Tv$$​ rotates the vector $$v$$​ in the direction of the eigen vector. Then, $$\Lambda(Q^Tv)$$​ stretches the vector $$v$$​. Finally, $$Q (\Lambda Q^Tv)$$​ rotates $$v$$​ back to its original direction.

Note: Eigen Decomposition may not be unique.