InnerProduct Layer是全連線層，CNN中常常出現在分類網路的末尾，將卷積全連線化或分類輸出結果，當然它的用處很多，不只是分類網路中

首先我們先看一下 InnerProductParameter

message InnerProductParameter {
	  optional uint32 num_output = 1; // The number of outputs for the layer   //輸出的個數
	  optional bool bias_term = 2 [default = true]; // whether to have bias terms  //是否使用bias
	  optional FillerParameter weight_filler = 3; // The filler for the weight  //全連線權重
	  optional FillerParameter bias_filler = 4; // The filler for the bias   //bias權重
	
	  // The first axis to be lumped into a single inner product computation;
	  // all preceding axes are retained in the output.
	  // May be negative to index from the end (e.g., -1 for the last axis).
	  //將第一個軸集中進行單個內積計算
	  optional int32 axis = 5 [default = 1];
	  // Specify whether to transpose the weight matrix or not.
	  // If transpose == true, any operations will be performed on the transpose
	  // of the weight matrix. The weight matrix itself is not going to be transposed
	  // but rather the transfer flag of operations will be toggled accordingly.
	  // 是否要轉置權矩陣。如果true，則對權矩陣執行轉置操作，注意權重矩陣本身不會被置換，而是操作傳輸標誌作相應切換
	  optional bool transpose = 6 [default = false];
}
 

InnerProduct layer 在prototxt裡面的書寫：

layer {
	  name: "fc"
	  type: "InnerProduct"  
	  bottom: "fc_in"
	  top: "fc_out"
	  
	  param {  # 權重學習引數
		    lr_mult: 1  # 學習率
		    decay_mult: 1
	  }
	  param {  # bias 學習引數
		    lr_mult: 2  # 一般情況，bias 學習率是權重學習率的兩倍.
		    decay_mult: 0
	  }
	  
	  inner_product_param {
		    num_output: 1000  # 輸出單元個數 
		    weight_filler {  # 權重初始化方法
		      type: "gaussian"
		      std: 0.005
		    }
		    bias_filler {  # bias 初始化方法
		      type: "constant"
		      value: 0.1
		    }
	 }
}
 

有些網路在實現時，常常用卷積來代替全連線層的功能，如MobileNet裡面：

layer {
	  name: "fc7"
	  type: "Convolution"
	  bottom: "pool6"
	  top: "fc7"
	  param {
		    lr_mult: 1
		    decay_mult: 1
	  }
	  param {
		    lr_mult: 2
		    decay_mult: 0
	  }
	  convolution_param {
		    num_output: 1000
		    kernel_size: 1
		    weight_filler {
		      type: "msra"
		    }
		    bias_filler {
		      type: "constant"
		      value: 0
		    }
	}
}