Vectors and Linear Combinations

Vectors

We have n separate numbers $v_1、v_2、v_3,...,v_n$，that produces a n-dimensional vector $v$，and $v$ is represented by an arrow.
$$
v=\left[
\begin{matrix}
v_1 \
v_2 \
.\
.\
.\
v_n
\end{matrix}
\right] = (v_1,v_2,...,v_n)
$$

Two-dimensional vector ：$v = \left[\begin{matrix} v_1 \ v_2 \end{matrix}\right]$ and $w = \left[\begin{matrix} w_1 \ w_2 \end{matrix}\right]$

• Vector Addition : $v + w = \left[\begin{matrix} v_1 + w_1 \ v_2 + w_2\end{matrix}\right]$
• Scalar Multiplication : $cv = \left[\begin{matrix} cv_1 \ cv_2 \end{matrix}\right]$，c is scalar.

Linear Combinations

Multiply $v$ by $c$ and multiply $w$ by $d$，the sum of $cv$ and $dw$ is a linear combination : $cv + dw$.

We can visualize $v + w$ using arrows，for example：


The combinations can fill Line、Plane 、or 3-dimensional space:

• The combinations $cu$ fill a line through origin.
• The combinations $cu + dv$ fill a plane throught origin
• The combinations $cu + dv +ew$ fill three-dimensional space throught origin.


Lengths and Dot Products

**Dot Product/ Inner Product : ** $v \cdot w = v_1w_1 + v_2w_2$，where $v = (v_1, v_2) $ and $w=(w_1, w_2)$ ，the dot product $w \cdot v$ equals $v \cdot w$

Length : $||v|| = \sqrt{v \cdot v} = (v_1^2 + v_2^2 + v_3^2 +...+ v_n²⁾{1/2}$

Unit vector : $u = v /||v||$ is a unit vector in the same direction as $v$，length =1

**Perpendicular vector : ** $v \cdot w = 0$

**Cosine Formula : ** if $v$ and $w$ are nonzero vectors then $\frac{v \cdot w}{||v|| \ ||w||} = cos \theta$ , $\theta$ is the angle between $v$ and $w$

Schwarz Inequality : $|v \cdot w| \leq ||v|| \ ||w||$

Triangel Inequality : $||v + w|| \leq ||v|| + ||w||$

Matrices

1、$A = \left[ \begin{matrix} 1 & 2 \ 3 & 4 \ 5 & 6 \end{matrix}\right]$ is a 3 by 2 matrix : m=2 rows and n=2 columns

2、$Ax = b $ is a linear combination of the columns A

3、 Combination of the vectors : $Ax = x_1\left[ \begin{matrix} 1 \ -1 \ 0 \end{matrix} \right] + x_2\left[ \begin{matrix} 0 \ 1 \ -1 \end{matrix} \right] + x_3\left[ \begin{matrix} 0 \ 0 \ 1 \end{matrix} \right] = \left[ \begin{matrix} x_1 \ x_2-x_1 \ x_3-x_2 \end{matrix} \right]$

4、Matrix times Vector : $Ax = \left[ \begin{matrix} 1&0&0\ -1&1&0 \ 0&-1&1 \end{matrix} \right] \left[ \begin{matrix} x_1\ x_2 \ x_3 \end{matrix} \right]= \left[ \begin{matrix} x_1 \ x_2-x_1 \ x_3-x_2 \end{matrix} \right] $

5、Linear Equation : Ax = b --> $\begin{matrix} x_1 = b_1 \ -x_1 + x_2 = b_2 \ -x_2 + x_3 = b_3 \end{matrix}$

6、Inverse Solution : $x = A^{-1}b$ -- > $\begin{matrix} x_1 = b_1 \ x_2 = b_1 + b_2 \ x_3 =b_1 + b_2 + b_3 \end{matrix}$, when A is an invertible matrix

7、Independent columns : Ax = 0 has one solution, A is an invertible matrix, the column vectors of A are independent. (example: $u,v,w$ are independent，No combination except $0u + 0v + 0w = 0$ gives $b=0$)

8、Dependent columns : Cx = 0 has many solutions, C is a singular matrix, the column vectors of C are dependent. (example: $u,v,w^$ are dependent，other combinations like $au + cv + dw^$ gives $b=0$)