多變數線性迴歸featureNormalize gradientDescentMulti normalEqn computeCostMulti
OptionalExercises
1.多變數線性迴歸
本部分使用多變數線性迴歸來預測房價。假設你想賣房子並想知道一個合適的市場價應該是多少。一種方式就是如這樣收集最近的房價並做一個房價模型。Ex1data2.txt包含了Portland、Oregon等地房價作為訓練集。第一列是房子尺寸(平方英尺),第二列是臥室的數量,第三列是房子的價格。
Ex1_multi.m已經為你準備好了來完成這個練習。
1.1FeatureNormalization
ex1_multi.m一開始會載入並顯示一些資料集中的值。通過看這些值,注意到房價是臥室數的近1000倍。當特徵之間的量級不一樣,第一個特徵的規模能夠使梯度下降收斂的更快。
你的任務是完成featureNormalize.m中的code來:
A. 資料集中的每個特徵值減去他們的平均值。
B. 減後,用這些值除他們各自的標準差。
標準差是一種表示一個特徵的波動範圍的測量方式;這是一種可供選擇的計算值的範圍(max-min)。Octave/matlab中你可以使用‘std’命令來計算標準差。例如,featureNormalize.m中X(:,1)的值包含所有的x1(房價)的值,所以std(X(:,1))計算了所以房價的標準差。此時,額外的列x0=1還沒有加入到X中(詳細看ex1_multi.m)。
你會為所有特徵都執行這種操作,因此你的code應該可用於所有尺寸的資料集。注意到矩陣X的每一列對應一個特徵。
function [X_norm, mu, sigma] = featureNormalize(X) %FEATURENORMALIZE Normalizes the features in X % FEATURENORMALIZE(X) returns a normalized version of X where % the mean value of each feature is 0 and the standard deviation % is 1. This is often a good preprocessing step to do when % working with learning algorithms. % You need to set these values correctly X_norm = X; mu = zeros(1, size(X, 2)); sigma = zeros(1, size(X, 2)); % ====================== YOUR CODE HERE ====================== % Instructions: First, for each feature dimension, compute the mean % of the feature and subtract it from the dataset, % storing the mean value in mu. Next, compute the % standard deviation of each feature and divide % each feature by it's standard deviation, storing % the standard deviation in sigma. % % Note that X is a matrix where each column is a % feature and each row is an example. You need % to perform the normalization separately for % each feature. % % Hint: You might find the 'mean' and 'std' functions useful. % mu=mean(X); % 1 x n; sigma=std(X); % 1 x n; mu_temp=((mu')*ones(1,m))'; % m x n sigma_temp=(sigma'*ones(1,m))' ; % m x n X_norm=(X-mu_temp)./sigma_temp; % ============================================================ end
1.2GradientDescent
在之前已經在單變量回歸問題使用了梯度下降。不同之處是,這裡多了一個變數。假設函式和批量梯度下降更新準則沒有變。需要完成computeCostMulti.m和gradientDescentMulti.m中的程式碼用於多變數線性迴歸。確保你的程式碼支撐任意特徵值並很好的向量化了。可以使用‘size(X,2)’來看資料中有多少個特徵。
function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters)
%GRADIENTDESCENTMULTI Performs gradient descent to learn theta
% theta = GRADIENTDESCENTMULTI(x, y, theta, alpha, num_iters) updates theta by
% taking num_iters gradient steps with learning rate alpha
% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);
for iter = 1:num_iters
% ====================== YOUR CODE HERE ======================
% Instructions: Perform a single gradient step on the parameter vector
% theta.
%
% Hint: While debugging, it can be useful to print out the values
% of the cost function (computeCostMulti) and gradient here.
%
predictions = X*theta;
errors = (predictions-y);
sums = X'*errors;
delta = 1/m*sums;
theta = theta - alpha*delta;
% ============================================================
% Save the cost J in every iteration
J_history(iter) = computeCostMulti(X, y, theta);
end
end
function J = computeCostMulti(X, y, theta)
%COMPUTECOSTMULTI Compute cost for linear regression with multiple variables
% J = COMPUTECOSTMULTI(X, y, theta) computes the cost of using theta as the
% parameter for linear regression to fit the data points in X and y
% Initialize some useful values
m = length(y); % number of training examples
% You need to return the following variables correctly
J = 0;
% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta
% You should set J to the cost.
m = size(X,1);
predictions = X*theta;
errors = predictions-y;
J = 1/(2*m)*(errors)'*errors;
% =========================================================================
end
注意,在多變數情況下,cost函式也能被寫為以下向量格式:
1.2.1Optional(ungraded)exercise:selecting learning raets
本部分中,對於資料集使用不同的學習速率並尋找最快收斂的學習速率。通過修改ex1_multi.m和改變部分關於學習速率的程式碼來改變學習速率。
ex1_multi.m中的下一部分會呼叫你的gradientDescentMulti.m函式,並在選擇的學習速率下迭代50次。函式還會在向量J中返回J(theta)的歷史值。最後一次迭代之後,畫出J(theta)相對於迭代次數的J的值。如果你選擇一個好的學習速率,你畫出來的應類似圖4.如果你的圖看起來非常不同,尤其是如果你的J(theta)的值增加或者波動,修改你的學習速率並再次嘗試。我們推薦以log的規模,3倍增加的形式來選學習速率(如0.3,0.1,0.03,0.01)。
1.3normalEquations
在課程視訊中, 學習到了和線性迴歸相似的解法是使用這個公式,不需要任何特徵scaling,並且你會在一次計算中得到一個額外的解:這裡也沒有像梯度下降中的“迴圈直到收斂”。完成normalEqn.m中的程式碼使用上述的公式計算θ。記得你不用縮放你的特徵,我們依舊需要加一個全1列在X中,來獲得截斷項(theta_0)。
function [theta] = normalEqn(X, y)
%NORMALEQN Computes the closed-form solution to linear regression
% NORMALEQN(X,y) computes the closed-form solution to linear
% regression using the normal equations.
theta = zeros(size(X, 2), 1);
% ====================== YOUR CODE HERE ======================
% Instructions: Complete the code to compute the closed form solution
% to linear regression and put the result in theta.
%
% ---------------------- Sample Solution ----------------------
theta = pinv( X' * X ) * X' * y;
% ------------------------------------------------------------
% ============================================================
end
%% Machine Learning Online Class
% Exercise 1: Linear regression with multiple variables
%
% Instructions
% ------------
% This file contains code that helps you get started on the
% linear regression exercise.
%
% You will need to complete the following functions in this
% exericse:
%
% warmUpExercise.m
% plotData.m
% gradientDescent.m
% computeCost.m
% gradientDescentMulti.m
% computeCostMulti.m
% featureNormalize.m
% normalEqn.m
%
% For this part of the exercise, you will need to change some
% parts of the code below for various experiments (e.g., changing
% learning rates).
%
%% Initialization
%% ================ Part 1: Feature Normalization ================
%% Clear and Close Figures
clear ; close all; clc
fprintf('Loading data ...\n');
%% Load Data
data = load('ex1data2.txt');
X = data(:, 1:2);
y = data(:, 3);
m = length(y);
% Print out some data points
fprintf('First 10 examples from the dataset: \n');
fprintf(' x = [%.0f %.0f], y = %.0f \n', [X(1:10,:) y(1:10,:)]');
fprintf('Program paused. Press enter to continue.\n');
pause;
% Scale features and set them to zero mean
fprintf('Normalizing Features ...\n');
[X mu sigma] = featureNormalize(X);
% Add intercept term to X
X = [ones(m, 1) X];
%% ================ Part 2: Gradient Descent ================
% ====================== YOUR CODE HERE ======================
% Instructions: We have provided you with the following starter
% code that runs gradient descent with a particular
% learning rate (alpha).
%
% Your task is to first make sure that your functions -
% computeCost and gradientDescent already work with
% this starter code and support multiple variables.
%
% After that, try running gradient descent with
% different values of alpha and see which one gives
% you the best result.
%
% Finally, you should complete the code at the end
% to predict the price of a 1650 sq-ft, 3 br house.
%
% Hint: By using the 'hold on' command, you can plot multiple
% graphs on the same figure.
%
% Hint: At prediction, make sure you do the same feature normalization.
%
fprintf('Running gradient descent ...\n');
% Choose some alpha value
alpha = 0.01;
num_iters = 400;
% Init Theta and Run Gradient Descent
theta = zeros(3, 1);
[theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters);
% Plot the convergence graph
figure;
plot(1:numel(J_history), J_history, '-b', 'LineWidth', 2);
xlabel('Number of iterations');
ylabel('Cost J');
% Display gradient descent's result
fprintf('Theta computed from gradient descent: \n');
fprintf(' %f \n', theta);
fprintf('\n');
% Estimate the price of a 1650 sq-ft, 3 br house
% ====================== YOUR CODE HERE ======================
% Recall that the first column of X is all-ones. Thus, it does
% not need to be normalized.
% price = 0; % You should change this
price = [1 (([1650 3]-mu) ./ sigma)] * theta ;
% ============================================================
fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
'(using gradient descent):\n $%f\n'], price);
fprintf('Program paused. Press enter to continue.\n');
pause;
%% ================ Part 3: Normal Equations ================
fprintf('Solving with normal equations...\n');
% ====================== YOUR CODE HERE ======================
% Instructions: The following code computes the closed form
% solution for linear regression using the normal
% equations. You should complete the code in
% normalEqn.m
%
% After doing so, you should complete this code
% to predict the price of a 1650 sq-ft, 3 br house.
%
%% Load Data
data = csvread('ex1data2.txt');
X = data(:, 1:2);
y = data(:, 3);
m = length(y);
% Add intercept term to X
X = [ones(m, 1) X];
% Calculate the parameters from the normal equation
theta = normalEqn(X, y);
% Display normal equation's result
fprintf('Theta computed from the normal equations: \n');
fprintf(' %f \n', theta);
fprintf('\n');
% Estimate the price of a 1650 sq-ft, 3 br house
% ====================== YOUR CODE HERE ======================
% price = 0; % You should change this
price = [1 1650 3] * theta ;
% ============================================================
fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
'(using normal equations):\n $%f\n'], price);
-----------------
作者:思念銘心
來源:CSDN
原文:https://blog.csdn.net/sinat_38930882/article/details/77934734
版權宣告:本文為博主原創文章,轉載請附上博文連結!