神經網路常用啟用函式及其應用舉例
阿新 • • 發佈:2019-01-26
假設神經元的輸入是一個4維的向量 X,其中的值為0或者1,並且只取4中樣本:
- x1=[1,0,0,0]
- x2=[0,1,0,0]
- x3=[0,0,1,0]
- x4=[0,0,0,1]
對這一組樣本,採用一個很小的隨機數來模擬生成20000個樣本,同時,將神經元的4個輸出值對映到[0,1],即4個分類分別是[0.25,0.5,0.75,1]。
程式碼實現如下:
#include <iostream> #include <vector> using namespace std; vector<vector<double>> inputData; vector<double> weight; double actual_output; //網路的實際輸出 vector<double> input; //生成numofSample個樣本,一共分為numofLable類,每個樣本的標籤是當前樣本的下標i%numofLable void getSamplesData(int numofSamples, int numofLable) { const int iterations = numofSamples; // 樣本數量 for (int i = 0; i < iterations; i++) { int index = i % numofLable; vector<double> dvect(numofLable, 0); dvect[index] = 1; for (size_t i = 0; i != dvect.size(); i++) { dvect[i] += (5e-3*rand() / RAND_MAX - 2.5e-3); } inputData.push_back(dvect); } } //初始化初始權重,包括4個節點和1個偏置,隨機設定為0~0.05之間的一個值 void intialWeight() { cout << "初始化權重為:" << endl; // 4個連結和一個偏置w0 for (int i = 0; i != 5; i++) { weight.push_back(0.05*rand() / RAND_MAX); cout << weight[i] << endl; } } //前向計算,輸入時資料層的4維的訓練資料,actual_output是網路當前的輸出 void cmtForward(const vector<double>& inVect) { double dsum = weight[4];//先把偏置加上 for (size_t i = 0; i != inVect.size(); i++) { dsum += (inVect[i] * weight[i]); } actual_output = 1 / (1 + exp(-1 * dsum)); // S函式的非線性變換 //actual_output = 2 * (1 / (1 + exp(-2 * dsum))) - 1; //T函式的非線性變換 //dsum > 0 ? actual_output = dsum : actual_output = 0; //R函式的非線性變換 } //權重調整,相當於後向計算,輸入時訓練資料和期望的輸出 void updataWeight(const vector<double>& inVect, const double true_output) { double learnRate = 0.05; // 權重更新引數 for (size_t i = 0; i != weight.size() - 1; i++) { weight[i] += (learnRate*(true_output - actual_output)*actual_output*(1 - actual_output)*inVect[i]); } // w0單獨計算 weight[4] += (learnRate*(true_output - actual_output)*actual_output*(1 - actual_output) * 1); } void main() { getSamplesData(20000, 4); //生成20000個數據,分為4類 intialWeight(); //初始化權重,隨機生成5個0~0.05之間的數值 //執行20000次迭代 for (int i = 0; i < 20000; i++) { input = inputData[i]; double lable = (double)(i % 4) / 4 + 0.25; //資料標籤轉換到0~1 cmtForward(input); //前向傳播計算 updataWeight(input, lable); //權重調整 if (i % 111 == 0) { cout << "當前執行第 " << i << " 次迭代" << endl; cout << "輸入資料: [ " << input[0] << "," << input[1] << "," << input[2] << "," << input[3] << " ]" << " " << "網路輸出:" << actual_output << endl << endl; } } cout << "優化後的權重係數:[ " << weight[0] << "," << weight[1] << "," << weight[2] << "," << weight[3] << " ]" << " 偏置: " << weight[4] << endl; system("pause"); }
在前向計算cmtForward函式中,使用S函式,註釋掉的分別是T函式和R函式。
S函式的迭代實現結果:
T函式的迭代實現結果:
R函式的迭代實現結果: