本篇博文主要結合前兩篇的knn演算法理論部分knn理論理解（一）和knn理論理解（二），做一個KNN的實現，主要是根據《機器學習實戰》這本書的內容，一個非常經典有趣的例子就是使用knn最近鄰演算法來實現對手寫數字的識別，下面將給出Python程式碼，儘量使用詳盡的解釋和註解讓大家更好地理解每一步的目的和方法，歡迎大家一起交流學習~~~

    我們使用的training資料儲存在trainingDigits當中，我們總共使用了100個樣本點資料，從0到9總共十個數字，每個數字有十個手寫資料，所以我們準備了總共100個樣本點來作為基準計算樣本；

而每個手寫數字則是以文字的形式儲存，每一個手寫數字都被提前進行了二值化處理，即只是用0,1兩個數字來表現手寫數字，如下所示：

這樣做的目的顯然可以方便我們將其轉化為特徵行向量進行計算，同時能夠儲存足夠的原資訊量用來進行數字識別；

而測試資料則是儲存在testDigits檔案當中，我們準備了50個樣本點用來測試識別分類效果，十個數字，每個數字有五個測試點，所以我們總共有50個樣本資料；（這裡我們要注意一下檔案的命名格式）

下面給出Python實現程式碼，程式碼主要包括三大塊吧，每一塊主要的功能分別是：

第一塊的目的是要將原本32*32大小的矩陣數字轉化為1*1024的行向量，第二塊是測試分類模組程式（獲得handwritinglabels數字標籤列表 將訓練資料儲存到大矩陣trainingMat當中去）（逐一讀取測試圖片並將其送到classify0分類器當中得出分類識別結果並最終列印結果），第三塊的目的是編寫分類主程式即knn演算法核心部分計算歐氏距離找出k近鄰點並將最多的那個作為測試樣本的識別結果；

<span style="font-size:14px;"># -*- coding: utf-8 -*-
#樣本是32*32的二值圖片，將其處理成1*1024的特徵行向量
#第一塊</span>
from numpy import *
import operator
from os import listdir

def img2vector(filename):
    returnVect = zeros((1,1024))   <span style="font-size:14px;">#產生一個數組將來用於儲存圖片資料</span>
    fr = open(filename)
    for i in range(32):
        lineStr = fr.readline()    <span style="font-size:14px;">#讀取第一行的32個二值資料</span>
        for j in range(32):
            returnVect[0,32*i+j] = int(lineStr[j])   <span style="font-size:14px;">#以32為一個迴圈 逐漸填滿1024個</span>
    return returnVect


<span style="font-size:14px;">#第二塊</span>
def handwritingClassTest():
    <span style="font-size:14px;">#載入訓練集到大矩陣trainingMat</span>
    hwLabels = []
    trainingFileList = listdir('C:\\Anaconda\\trainingDigits')           <span style="font-size:14px;">#os模組中的listdir('str')可以讀取目錄str下的所有檔名，返回一個字串列表</span>
    m = len(trainingFileList)                 <span style="font-size:14px;">#m表示總體訓練樣本個數</span>
    trainingMat = zeros((m,1024))             <span style="font-size:14px;">#用來存放m個樣本</span>
    for i in range(m):
        fileNameStr = trainingFileList[i]                  <span style="font-size:14px;">#訓練樣本的命名格式：1_120.txt 獲取檔名</span>
        fileStr = fileNameStr.split('.')[0]                <span style="font-size:14px;">#string.split('str')以字元str為分隔符切片，返回list，這裡去list[0],得到類似1_120這樣的</span>
        classNumStr = int(fileStr.split('_')[0])           <span style="font-size:14px;">#以_切片，得到1，即數字類別</span>
        hwLabels.append(classNumStr)                    <span style="font-size:14px;">#這樣hwLabels列表中儲存了m個樣本點的所有類別</span>
        trainingMat[i,:] = img2vector('C:\\Anaconda\\trainingDigits\\%s' % fileNameStr)    <span style="font-size:14px;">#將每個樣本存到m*1024矩陣</span>
        
    <span style="font-size:14px;">#逐一讀取測試圖片，同時將其分類 </span>  
    testFileList = listdir('C:\\Anaconda\\testDigits')     <span style="font-size:14px;">#測試檔名列表   </span>
    errorCount = 0.0
    mTest = len(testFileList)
    for i in range(mTest):
        fileNameStr = testFileList[i]            <span style="font-size:14px;">#獲取此刻要測試的檔名</span>
        fileStr = fileNameStr.split('.')[0]     
        classNumStr = int(fileStr.split('_')[0])          <span style="font-size:14px;">#數字標籤分類 重新生成的classNumStr 並沒有使用上面已有的classNumStr</span>
        vectorUnderTest = img2vector('C:\\Anaconda\\testDigits\\%s' % fileNameStr)  <span style="font-size:14px;">#將要測試的數字轉化為一行向量</span>
        classifierResult = classify0(vectorUnderTest, trainingMat, hwLabels, 3)   <span style="font-size:14px;">#傳參</span>
        print "the classifier came back with: %d, the real answer is: %d" % (classifierResult, classNumStr)
        if (classifierResult != classNumStr): errorCount += 1.0
    print "\nthe total number of errors is: %d" % errorCount
    print "\nthe total error rate is: %f" % (errorCount/float(mTest))


<span style="font-size:14px;">#第三塊
#分類主程式，計算歐式距離，選擇距離最小的前k個，返回k箇中出現頻次最高的類別作為分類別
#inX是所要測試的向量
#dataSet是訓練樣本集，一行對應一個樣本，dataSet對應的標籤向量為labels
#k是所選的最近鄰數</span>
def classify0(inX, dataSet, labels, k):      <span style="font-size:14px;">#引數和上面的一一對應</span>
    dataSetSize = dataSet.shape[0]                       <span style="font-size:14px;">#shape[0]得出dataSet的行數，即訓練樣本個數
    diffMat = tile(inX, (dataSetSize,1)) - dataSet       #tile(A,(m,n))將陣列A作為元素構造m行n列的陣列 100*1024的陣列
    sqDiffMat = diffMat**2
    sqDistances = sqDiffMat.sum(axis=1)                  #array.sum(axis=1)按行累加，axis=0為按列累加
    distances = sqDistances**0.5                        #就是sqrt的結果
    sortedDistIndicies = distances.argsort()             #array.argsort()，得到每個元素按次排序後分別在原陣列中的下標 從小到大排列
    classCount={}                                        #sortedDistIndicies[0]表示排序後排在第一個的那個數在原來陣列中的下標
    for i in range(k):
        voteIlabel = labels[sortedDistIndicies[i]]   #最近的那個在原來陣列中的下標位置
        classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1 #一個字典從無到有的生成過程  get(key,x)從字典中獲取key對應的value，沒有key的話返回0
    #classCount的形式:{5:3,0:6,1:7,2:1}
    sortedClassCount = sorted(classCount.iteritems(), key=operator.itemgetter(1), reverse=True) #sorted()函式，按照第二個元素即value的次序逆向（reverse=True）排序
                                                         #sorted第一個引數表示要排序的物件 iteritems代表鍵值對 
    return sortedClassCount[0][0]                        #經過sorted後的字典變成了[(),()]形式的鍵值對列表</span>
 

import knn

knn.handwritingClassTest()
the classifier came back with: 0, the real answer is: 0
the classifier came back with: 0, the real answer is: 0
the classifier came back with: 0, the real answer is: 0
the classifier came back with: 0, the real answer is: 0
the classifier came back with: 0, the real answer is: 0
the classifier came back with: 1, the real answer is: 1
the classifier came back with: 1, the real answer is: 1
the classifier came back with: 1, the real answer is: 1
the classifier came back with: 1, the real answer is: 1
the classifier came back with: 1, the real answer is: 1
the classifier came back with: 2, the real answer is: 2
the classifier came back with: 2, the real answer is: 2
the classifier came back with: 2, the real answer is: 2
the classifier came back with: 2, the real answer is: 2
the classifier came back with: 2, the real answer is: 2
the classifier came back with: 3, the real answer is: 3
the classifier came back with: 3, the real answer is: 3
the classifier came back with: 3, the real answer is: 3
the classifier came back with: 3, the real answer is: 3
the classifier came back with: 3, the real answer is: 3
the classifier came back with: 4, the real answer is: 4
the classifier came back with: 4, the real answer is: 4
the classifier came back with: 4, the real answer is: 4
the classifier came back with: 4, the real answer is: 4
the classifier came back with: 4, the real answer is: 4
the classifier came back with: 5, the real answer is: 5
the classifier came back with: 5, the real answer is: 5
the classifier came back with: 5, the real answer is: 5
the classifier came back with: 5, the real answer is: 5
the classifier came back with: 5, the real answer is: 5
the classifier came back with: 6, the real answer is: 6
the classifier came back with: 6, the real answer is: 6
the classifier came back with: 6, the real answer is: 6
the classifier came back with: 6, the real answer is: 6
the classifier came back with: 6, the real answer is: 6
the classifier came back with: 7, the real answer is: 7
the classifier came back with: 7, the real answer is: 7
the classifier came back with: 7, the real answer is: 7
the classifier came back with: 7, the real answer is: 7
the classifier came back with: 7, the real answer is: 7
the classifier came back with: 8, the real answer is: 8
the classifier came back with: 8, the real answer is: 8
the classifier came back with: 8, the real answer is: 8
the classifier came back with: 8, the real answer is: 8
the classifier came back with: 8, the real answer is: 8
the classifier came back with: 9, the real answer is: 9
the classifier came back with: 9, the real answer is: 9
the classifier came back with: 9, the real answer is: 9
the classifier came back with: 9, the real answer is: 9
the classifier came back with: 9, the real answer is: 9

the total number of errors is: 0

the total error rate is: 0.000000

我們發現準確率還是挺高的，沒有一個錯誤的，原因是我們使用的資料量比較小，恰巧沒有一個錯誤的；