多分類例題 - 手寫數字識別

提供的資料集包括5000張手寫數字0~9圖片及對應的正確數字值。其中，每一張圖片已被預處理成20 * 20 畫素的灰白圖片，並轉化成灰度存入到矩陣中。要求利用OVA演算法進行手寫數字識別。

繪製訓練集

本題的圖片繪製涉及灰度的一些內容，我並不瞭解。這裡使用coursera的程式碼進行繪製（因此也沒必要展示了），簡單觀察一下各個圖片。

設計思路

1. 由於輸出有0～9十種狀態，因此需要擬合10個目標函式，分別估計給定數字為0～9的可能性。最終取可能性最大的數字作為最終預測答案。
2. 由於每一個數據都是20*20畫素的灰白圖片，因此可以將其轉化1*400的向量。即每一個輸入值有400個屬性。
3. 由於matlab下標從0開始，因此在for迴圈中求0的目標函式不太方便。因此coursera提供的資料集中所有的y=0均用y=10代替。（其實沒必要這麼做，繞來繞去更加麻煩。還不如直接mod10）

訓練

首先寫出代價函式的計算函式，方便後期呼叫

function [J, grad] = lrCostFunction(theta, X, y, lambda)
%LRCOSTFUNCTION Compute cost and gradient for logistic regression with 
%regularization
%   J = LRCOSTFUNCTION(theta, X, y, lambda) computes the cost of using
%   theta as the parameter for regularized logistic regression and the
%   gradient of the cost w.r.t. to the parameters. 

% Initialize some useful values
m = length(y); % number of training examples

% You need to return the following variables correctly 
J = 0;
grad = zeros(size(theta));

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta.
%               You should set J to the cost.
%               Compute the partial derivatives and set grad to the partial
%               derivatives of the cost w.r.t. each parameter in theta
%
% Hint: The computation of the cost function and gradients can be
%       efficiently vectorized. For example, consider the computation
%
%           sigmoid(X * theta)
%
%       Each row of the resulting matrix will contain the value of the
%       prediction for that example. You can make use of this to vectorize
%       the cost function and gradient computations. 
%
% Hint: When computing the gradient of the regularized cost function, 
%       there're many possible vectorized solutions, but one solution
%       looks like:
%           grad = (unregularized gradient for logistic regression)
%           temp = theta; 
%           temp(1) = 0;   % because we don't add anything for j = 0  
%           grad = grad + YOUR_CODE_HERE (using the temp variable)
%

J = -1/m * sum(y .* log(sigmoid(X*theta))+(1-y) .* log(1-sigmoid(X*theta))) + lambda / (2*m) * sum(theta(2:end,:) .* theta(2:end,:));
temp = theta;
temp(1) = 0;
grad = 1/m * X' * (sigmoid(X*theta)-y) + lambda/m * temp;
% =============================================================

grad = grad(:);

end

 

然後在開始利用OVA訓練10個目標函式。利用for迴圈分別對1至10（還記得麼，數字0用10表示）求解目標函式。

function [all_theta] = oneVsAll(X, y, num_labels, lambda)
%ONEVSALL trains multiple logistic regression classifiers and returns all
%the classifiers in a matrix all_theta, where the i-th row of all_theta 
%corresponds to the classifier for label i
%   [all_theta] = ONEVSALL(X, y, num_labels, lambda) trains num_labels
%   logistic regression classifiers and returns each of these classifiers
%   in a matrix all_theta, where the i-th row of all_theta corresponds 
%   to the classifier for label i

% Some useful variables
m = size(X, 1);
n = size(X, 2);

% You need to return the following variables correctly 
all_theta = zeros(num_labels, n + 1);

% Add ones to the X data matrix
X = [ones(m, 1) X];

% ====================== YOUR CODE HERE ======================
% Instructions: You should complete the following code to train num_labels
%               logistic regression classifiers with regularization
%               parameter lambda. 
%
% Hint: theta(:) will return a column vector.
%
% Hint: You can use y == c to obtain a vector of 1's and 0's that tell you
%       whether the ground truth is true/false for this class.
%
% Note: For this assignment, we recommend using fmincg to optimize the cost
%       function. It is okay to use a for-loop (for c = 1:num_labels) to
%       loop over the different classes.
%
%       fmincg works similarly to fminunc, but is more efficient when we
%       are dealing with large number of parameters.
%
% Example Code for fmincg:
%
%     % Set Initial theta
%     initial_theta = zeros(n + 1, 1);
%     
%     % Set options for fminunc
%     options = optimset('GradObj', 'on', 'MaxIter', 50);
% 
%     % Run fmincg to obtain the optimal theta
%     % This function will return theta and the cost 
%     [theta] = ...
%         fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), ...
%                 initial_theta, options);
%
    
for idx=1:10
    Init_theta = zeros(size(X,2),1);
    options = optimset('GradObj','on','MaxIter',50);
    [theta,~,~]= fmincg(@(t)lrCostFunction(t,X,(y==idx),lambda),Init_theta,options);
    theta = theta';
    all_theta(idx,:) = theta;

end


% =========================================================================


end

 

呼叫方法：

fprintf('\nTraining One-vs-All Logistic Regression...\n')

lambda = 0.1;
[all_theta] = oneVsAll(X, y, num_labels, lambda);

fprintf('Program paused. Press enter to continue.\n');
pause;

結果判定

下面這段程式碼來自coursera，它只能判斷經驗誤差，而不能判斷泛化誤差。

pred = predictOneVsAll(all_theta, X);

fprintf('\nTrraining Set Accuracy: %f\n', mean(double(pred == y)) * 100);