−θTx" role="presentation" style="position: relative;"> 11+e−θTx $\frac{1}{1+e^{-{\theta }^{T}x}}$$\frac{1}{1 + e^{-\theta^Tx}}$, yet we sometimes call it a sigmoid (logistic) activation function. In this situation, our “theta” parameters are sometimes called “weights”.
Visually, a simplistic representation looks like:

$$\begin{bmatrix} x_0 \\ x_1 \\ x_2 \end{bmatrix} →[ \qquad ]→h_θ(x)$$

three layers: input layer / hidden layer / output layer
$a_{i}^{(j)}$ : activation unit i in layer j
$\Theta^{(j)}$ : Matrix that controls function mapping from j-th layer to (j+1)-th layer
If layer j has $s_{j}$ units, layer j+1 has $s_{j+1}$ units, then size of $\Theta^{(j)}$ is $s_{j+1}*(s_{j}+1)$
$L$ : Number of Layers
$s_l$ : Number of units in l-th layer
Number of Inputs: the dimension of features in $x^{(i)}$
Binary Classification: 1 output unit
K-classes Classification: K output unit

2. Forward Propagation

1) Add $a_x^{(0)}=1$ first
2) $z_{x+1}=\Theta^{(x)}a_x$
3) $a_{x+1}=g(z_{x+1})$ — g(x) : Sigmoid

3. Cost Function

Excluding Bias Term

4. Backpropagation Algorithm

$\delta_j^{(l)}$ error of node j in layer l, then

$$\delta^{(L)}=a^{(L)}-y\\ \delta^{(i)}=(\Theta^{(i)})^T\delta^{(i+1)}.*g'(z^{(i)}) \qquad (i!=L,i!=1)$$
where $g'(z^{(i)})=a^{(i)}.*(1-a^{(i)})$

One thing to note: use one training set to train the model at one time!

5. Unrolling Parameters

Enroll them to vectors/Get back:

6.Gradient Checking

When learning, turn off gradient checking!!!

7. Random Initialization

8.Network Architecture

one hidden layer/
more than one hidden layer with same number of units