【Python例項第14講】普通判別分析與縮水判別分析
阿新 • • 發佈:2018-11-29
機器學習訓練營——機器學習愛好者的自由交流空間(qq 群號:696721295)
這個例子說明在判別分析裡使用縮水(shrinkage
)的方法,可以提高分類的準確率。所謂“縮水”,是指減少預測的特徵。我們使用的資料集是模擬資料,你也可以在真實資料集上驗證縮水判別分析的分類效果。
例項詳解
首先,匯入必需的庫。
from __future__ import division import numpy as np import matplotlib.pyplot as plt from sklearn.datasets import make_blobs from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
準備工作
n_train = 20 # samples for training
n_test = 200 # samples for testing
n_averages = 50 # how often to repeat classification
n_features_max = 75 # maximum number of features
step = 4 # step size for the calculation
函式generate_data()
函式generate_data()用來生成模擬資料集。它有兩個引數n_samples
, n_features
def generate_data(n_samples, n_features): """Generate random blob-ish data with noisy features. This returns an array of input data with shape `(n_samples, n_features)` and an array of `n_samples` target labels. Only one feature contains discriminative information, the other features contain only noise. """ X, y = make_blobs(n_samples=n_samples, n_features=1, centers=[[-2], [2]]) # add non-discriminative features if n_features > 1: X = np.hstack([X, np.random.randn(n_samples, n_features - 1)]) return X, y
普通判別分析 v.s. 縮水判別分析
scikit-learn的線性判別分析函式LinearDiscriminantAnalysis
使用一個線性邊界的分類器,它使用貝葉斯規則的類條件密度擬合數據。模型假設所有的類有相同的協方差陣,對每一個類擬合一個高斯密度。LinearDiscriminantAnalysis的主要引數solver
是一個字串,指定使用的解形式。在這裡,它取lsqr
, 表示最小二乘解,可以與縮水法結合。引數shrinkage
預設取值None
, 表示沒有縮水。在這裡,它取auto
, 表示使用Ledoit-Wolf(估計縮水的協方差陣)自動縮水。
acc_clf1, acc_clf2 = [], []
n_features_range = range(1, n_features_max + 1, step)
for n_features in n_features_range:
score_clf1, score_clf2 = 0, 0
for _ in range(n_averages):
X, y = generate_data(n_train, n_features)
clf1 = LinearDiscriminantAnalysis(solver='lsqr', shrinkage='auto').fit(X, y)
clf2 = LinearDiscriminantAnalysis(solver='lsqr', shrinkage=None).fit(X, y)
X, y = generate_data(n_test, n_features)
score_clf1 += clf1.score(X, y)
score_clf2 += clf2.score(X, y)
acc_clf1.append(score_clf1 / n_averages)
acc_clf2.append(score_clf2 / n_averages)
features_samples_ratio = np.array(n_features_range) / n_train
視覺化分類效果
plt.plot(features_samples_ratio, acc_clf1, linewidth=2,
label="Linear Discriminant Analysis with shrinkage", color='navy')
plt.plot(features_samples_ratio, acc_clf2, linewidth=2,
label="Linear Discriminant Analysis", color='gold')
plt.xlabel('n_features / n_samples')
plt.ylabel('Classification accuracy')
plt.legend(loc=1, prop={'size': 12})
plt.suptitle('Linear Discriminant Analysis vs. \
shrinkage Linear Discriminant Analysis (1 discriminative feature)')
plt.show()
閱讀更多精彩內容,請關注微信公眾號:統計學習與大資料