樸素貝葉斯演算法及應用

作業資訊

實驗目的

1.理解樸素貝葉斯演算法原理，掌握樸素貝葉斯演算法框架；
2.掌握常見的高斯模型，多項式模型和伯努利模型；
3.能根據不同的資料型別，選擇不同的概率模型實現樸素貝葉斯演算法；
4.針對特定應用場景及資料，能應用樸素貝葉斯解決實際問題。

實驗要求

1.實現高斯樸素貝葉斯演算法。
2.熟悉sklearn庫中的樸素貝葉斯演算法；
3.針對iris資料集，應用sklearn的樸素貝葉斯演算法進行類別預測。
4.針對iris資料集，利用自編樸素貝葉斯演算法進行類別預測。

實驗報告內容

1.對照實驗內容，撰寫實驗過程、演算法及測試結果；
2.程式碼規範化：命名規則、註釋；
3.分析核心演算法的複雜度；
4.查閱文獻，討論各種樸素貝葉斯演算法的應用場景；
5.討論樸素貝葉斯演算法的優缺點。

實驗程式碼

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from collections import Counter
import math
 

# data
def create_data():
    iris = load_iris()
    df = pd.DataFrame(iris.data, columns=iris.feature_names)
    df['label'] = iris.target
    df.columns = [
        'sepal length', 'sepal width', 'petal length', 'petal width', 'label'
    ]
    data = np.array(df.iloc[:100, :])
    # print(data)
    return data[:, :-1], data[:, -1]
 

X, y = create_data()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

X_test[0], y_test[0]

class NaiveBayes:
    def __init__(self):
        self.model = None
    # 數學期望
    @staticmethod
    def mean(X):
        return sum(X) / float(len(X))
    # 標準差（方差）
    def stdev(self, X):
        avg = self.mean(X)
        return math.sqrt(sum([pow(x - avg, 2) for x in X]) / float(len(X)))
    # 概率密度函式
    def gaussian_probability(self, x, mean, stdev):
        exponent = math.exp(-(math.pow(x - mean, 2) /
                              (2 * math.pow(stdev, 2))))
        return (1 / (math.sqrt(2 * math.pi) * stdev)) * exponent
    # 處理X_train
    def summarize(self, train_data):
        summaries = [(self.mean(i), self.stdev(i)) for i in zip(*train_data)]
        return summaries
    # 分類別求出數學期望和標準差
    def fit(self, X, y):
        labels = list(set(y))
        data = {label: [] for label in labels}
        for f, label in zip(X, y):
            data[label].append(f)
        self.model = {
            label: self.summarize(value)
            for label, value in data.items()
        }
        return 'gaussianNB train done!'
    # 計算概率
    def calculate_probabilities(self, input_data):
        # summaries:{0.0: [(5.0, 0.37),(3.42, 0.40)], 1.0: [(5.8, 0.449),(2.7, 0.27)]}
        # input_data:[1.1, 2.2]
        probabilities = {}
        for label, value in self.model.items():
            probabilities[label] = 1
            for i in range(len(value)):
                mean, stdev = value[i]
                probabilities[label] *= self.gaussian_probability(
                    input_data[i], mean, stdev)
        return probabilities
    # 類別
    def predict(self, X_test):
        # {0.0: 2.9680340789325763e-27, 1.0: 3.5749783019849535e-26}
        label = sorted(
            self.calculate_probabilities(X_test).items(),
            key=lambda x: x[-1])[-1][0]
        return label
    def score(self, X_test, y_test):
        right = 0
        for X, y in zip(X_test, y_test):
            label = self.predict(X)
            if label == y:
                right += 1
        return right / float(len(X_test))

model = NaiveBayes()

model.fit(X_train, y_train)

print(model.predict([4.4, 3.2, 1.3, 0.2]))

model.score(X_test, y_test)

from sklearn.naive_bayes import GaussianNB

clf = GaussianNB()
clf.fit(X_train, y_train)

clf.score(X_test, y_test)

clf.predict([[4.4, 3.2, 1.3, 0.2]])

from sklearn.naive_bayes import BernoulliNB, MultinomialNB # 伯努利模型和多項式模型

執行截圖

實驗小結