Singular Value Decomposition or SVD is generalization of factorization to non-square matrices. The factorizing of matrices is based upon stretch and rotation.
If A ( R m × n ) A (\mathbb{R}^{m \times n}) A ( R m × n )
A = U D V T , w h e r e U ∈ R m × m a n d o r t h o g o n a l . V ∈ R n × n a n d o r t h o g o n a l . D ∈ R m × n d i a g o n a l . O n l y d i a g o n a l e n t r i e s a r e n o n z e r o . A = UDV^T, where
\\
U \in \mathbb{R^{m \times m}} \hspace{0.2cm} and \hspace{0.1cm} orthogonal.
\\
V \in \mathbb{R^{n \times n}} \hspace{0.2cm} and \hspace{0.1cm} orthogonal.
\\
D \in \mathbb{R^{m \times n}} \hspace{0.2cm} diagonal. \hspace{0.1cm} Only \hspace{0.1cm} diagonal \hspace{0.1cm} entries \hspace{0.1cm} are \hspace{0.1cm} non \hspace{0.1cm} zero. A = U D V T , w h ere U ∈ R m × m an d or t h o g o na l . V ∈ R n × n an d or t h o g o na l . D ∈ R m × n d ia g o na l . O n l y d ia g o na l e n t r i es a re n o n zero . Elements of U U U A A T AA^T A A T
Elements of V V V A T A A^TA A T A
Non-zero elements of D D D λ ( A T A ) \sqrt{\lambda(A^TA)} λ ( A T A ) λ ( A T A ) \lambda(A^TA) λ ( A T A ) A T A A^TA A T A
