寫在前面:

此次作業是使用兩種方式來識別手寫數字，順序為邏輯迴歸、神經網路前向傳播（帶已經訓練好的引數）

1. 邏輯迴歸解決多分類問題

思路：對於每個數字，建立一個分類器（比如說1，則建立分類器識別該組資料是否是一），然後算出每個分類器的引數，將所有分類器（因為此處是數字，所以有十組引數）引數存在一個總的引數資料集中（此資料集就是訓練的結果，也就是說通過此資料集可以判斷一個新的手寫數字圖片是數字幾）

tips：原理就是將圖片的每個引數當做特徵，然後訓練模型來識別特徵值達到學習的目的，最終得到分類器（可以識別新數字的）

下面上原始碼：

'''
warning:
the range of initial data in matlab file is 1-10
but in this place ,the 10 is indicate 0

the data in ex3data1.mat is so many row which size is 1*400(has been compressed)
 
'''

# 此處不使用自己定義的梯度下降函式，使用scipy模組中自帶的minimize函式

import numpy as np
import matplotlib.pyplot as plt
import scipy.io as sio
from scipy.optimize import minimize

# the function which can plot one image(anyone) in pic_set

'''
minimize引數解釋：
- fun：要優化的函式(損失函式)
- method：優化方法
- jac：梯度向量
- x0：引數初始值(n,)(n,1)

(n,)(n,1)的區別
 
'''
def plot_an_image(X):
    pick_one = np.random.randint(len(X))
    image = X[pick_one,:]
    fig,ax = plt.subplots(figsize = (1,1))
    ax.imshow(image.reshape(20,20).T,cmap = 'gray_r')
    # 去掉刻度
    plt.xticks([])
    plt.yticks([])
    plt.show()

def plot_100_image(X):
    sample_index = np.random.choice(len(X),100)
    images 
 
= X[sample_index,:]
    # 後面兩個引數控制橫縱座標是否顯示
    fig,ax = plt.subplots(ncols=10,nrows=10,figsize = (8,8),sharex=True,sharey=True)
    for r in range(10):
        for c in range(10):
            ax[r][c].imshow(images[10*r+c].reshape(20,20).T,cmap = 'gray_r')
    # ax.imshow(image.reshape(20,20).T,cmap = 'gray_r')
    # 去掉刻度
    plt.xticks([])
    plt.yticks([])
    plt.show()

def sigmoid(z):
    return 1/(1+np.exp(-z))

# 注意此處的傳入引數順序是固定的，打亂會導致呼叫np模組內優化演算法出錯
def costFunction(theta,X,y,lamda):
    A = sigmoid(X@theta)

    first = (~y)* np.log(A)
    second = (1-y) * np.log(1-A)

    # 懲罰項
    # 兩種寫法都可以
    # reg = np.sum(np.power(theta[1:],2))*( lamda / 2 * len(X))
    reg = theta[1:]@theta[1:]*(lamda / 2 * len(X))

    return np.sum(first - second)/len(X) + reg
# 無需迴圈，使用np的內建方法
def gradient_reg(theta,X,y,lamda):
    reg = theta[1:]*(lamda/len(X))
    reg = np.insert(reg,0,values = 0,axis = 0)

    first = (X.T@(sigmoid(X@theta) - y))/len(X)

    return first + reg

def one_va_all(X,y,lamda,K):
    n = X.shape[1]
    # 使用二維陣列來儲存所有theta
    theta_all = np.zeros((K,n))

    for i in range(1,K+1):
        # 定義第i個分類器的引數
        theta_i = np.zeros(n,)
        # 判別式y==i是用來分辨是第幾類標籤
        res = minimize(fun = costFunction,
                       x0 = theta_i,
                       args = (X,y == i,lamda),
                       method = 'TNC',
                       jac = gradient_reg)
        theta_all[i-1,:] = res.x

    return theta_all

def predict(X,theta_final):
    h = sigmoid(X@theta_final.T)
    h_argmax = np.argmax(h,axis = 1)# 返回一行中最大值的索引
    return h_argmax + 1

data = sio.loadmat('./data_set/ex3data1.mat')
# print(data)
# print(type(data))
# print(data.keys())

raw_X  = data['X']
raw_y  = data['y']
# print(raw_X.shape)
# print(raw_X[1].shape)
# plot_an_image(raw_X)
# plot_100_image(raw_X)

X = np.insert(raw_X,0,values=1,axis = 1)
# 將y從二維陣列變成一維陣列
y = raw_y.flatten()

lamda = 1
K = 10
theta_final = one_va_all(X,y,lamda,K)
# for each in y:
#     print(each)

y_pred = predict(X,theta_final)
acc = np.mean(y == y_pred)
print(acc)

2.使用神經網路解決問題

思路：此處因為已經有訓練好的引數，所以直接前向傳播就好，此處有一個隱藏層，一個輸入層，一個輸出層，啟用函式使用sigmoid函式，輸入層傳入的引數是每個照片的圖片引數，然後輸出層是每個圖片在每一個數字判斷器中運算的結果（也就是概率），最大的那個就是預測出來的數值

下面上原始碼：

'''
natural network
use natural network to implement the recognization of hand write number
input the number image's attribute
and output the predict number of each image
'''

import numpy as np
import scipy.io as sio

def sigmoid(z):
    return 1/(1+np.exp(-z))



data = sio.loadmat('./data_set/ex3data1.mat')
raw_X = data['X']
raw_y = data['y']

X = np.insert(raw_X,0,values = 1,axis = 1)
# X.shape
y = raw_y.flatten()

# the theta is format by dictionary
theta = sio.loadmat('./data_set/ex3weights.mat')
# print(theta.keys())

# <Theta1> includes the attribute of input layer to hidden layer
# <Theta2> includes the attribute of hidden layer to output layer
theta1 = theta['Theta1']
theta2 = theta['Theta2']

# use sigmoid function as activate function
a1 = X
# obtain the number of hidden layer
z2 = X @ theta1.T
a2 = sigmoid(z2)

a2 = np.insert(a2,0,values = 1,axis = 1)
# obtain the number of output layer
z3 = a2 @ theta2.T
a3 = sigmoid(z3)

y_pred = np.argmax(a3,axis = 1)
y_pred += 1

acc = np.mean(y_pred==y)
print(acc)

以上

希望對大家有所幫助