一、lasso

二、前向逐步迴歸

   前向逐步迴歸演算法可以得到與lasso差不多的效果，但是前向逐步迴歸更加簡單。這是一種貪心演算法，在每一步儘量減少誤差。

三、實驗

1、Matlab實現

主程式

clear all;
clc;
%% 匯入資料
data = load('abalone.txt');
x = data(:,1:8);
y = data(:,9);
%% 處理資料
yMean = mean(y);
yDeal = y-yMean;
xMean = mean(x);
xVar = var(x,1);
[m,n] = size(x);
xDeal = zeros(m,n);
for i = 1:m
    for j = 1:n
        xDeal(i,j) = (x(i,j)-xMean(j))/xVar(j);
    end
end

%% 訓練
runtime  = 5000;%迭代的步數
eps = 0.001;%調整步長
wResult = stageWise(xDeal, yDeal, eps, runtime);

%% 根據wResult畫出收斂曲線
hold on 
xAxis = 1:runtime;
for i = 1:n
    plot(xAxis, wResult(:,i));
end
 

前向逐步迴歸函式

function [ wResult ] = stageWise( x, y, eps, runtime)
    [m,n] = size(x);%資料集的大小
    wResult = zeros(runtime, n);%最終的結果
    w = zeros(n,1);
    wMax = zeros(n,1);
    for i = 1:runtime
        ws = w'%輸出每一次計算出來的權重
        lowestError = inf;%定義最小值
        for j = 1:n
            for sign = -1:2:1
                wTest = w;%初始化
                wTest(j) = wTest(j)+eps*sign;%只改變一維變數
                yTest = x*wTest;
                %求誤差
                rssE = rssError(y, yTest);
                if rssE < lowestError%如果好，就替換
                    lowestError = rssE;
                    wMax = wTest;
                end
            end
        end
        w = wMax;
        wResult(i,:) = w;
    end
end
 

誤差函式

%% rssError函式主要是利用均方誤差
function [ error ] = rssError( y, yTest )
    yDis = y-yTest;%誤差
    [m,n] = size(yDis);
    %求平方
    for i = 1:m
        yDis(i) = yDis(i)^2;
    end
    error = sum(yDis);%求列和
end

2、收斂曲線