The Singular Value Decomposition of $A$ is:
$$A = U \Sigma V^{\text{T}} = \sigma_1 u_1 v_1^{\text{T}} + \ldots + \sigma_r u_r v_r^{\text{T}}$$
Reduced form of SVD, has $r$ $v$'s and $u$'s and $\sigma$'s. We can reduce $AV = U\Sigma$ to $AV_r = U_r \Sigma_r$ by removing parts that are sure to produce zeros ($r$ is the rank of the matrix). Now $\Sigma_r$ is square.
The principal components of $A$ are its singular vectors, the columns $u_j$ and $v_j$ of the orthogonal matrices $U$ and $V$. Principal Component Analysis (PCA) uses the largest $\sigma$'s connected to the first $u$'s and $v$'s to understand the information in a matrix of data. We are given a matrix $A$, and we extract its most important part $A_k$,
$$A_k = \sigma_1 u_1 v_1^{\text{T}} + \ldots + \sigma_k u_k v_k^{\text{T}}$$