L2正則化—tensorflow實現
阿新 • • 發佈:2019-02-18
L2正則化是一種減少過擬合的方法,在損失函式中加入刻畫模型複雜程度的指標。假設損失函式是J(θ) ,則優化的是J(θ)+λR(w) ,R(w)=∑ni=0|w2i| 。
在tensorflow中的具體實現過程如下:
#coding:utf-8
import tensorflow as tf
def get_weight(shape,lambda):
var = tf.Variable(tf.random_normal(shape),dtype=tf.float32)
tf.add_to_collection("losses",tf.contrib.layers.l2_regularizer(lambda )(var))#把正則化加入集合losses裡面
return var
x = tf.placeholder(tf.float32,shape=(None,2))
y_ = tf.placeholder(tf.float32,shape=(none,1))#真值
batcg_size = 8
layer_dimension = [2,10,10,10,1]#神經網路層節點的個數
n_layers = len(layer_dimension)#神經網路的層數
cur_layer = x
in_dimension = layer_dimension[0]
for i in range (1 ,n_layers):
out_dimension = layer_dimension[i]
weight = get_weight([in_dimension,out_dimension],0.001)
bias = tf.Variable(tf.constant(0.1,shape(out_dimension)))
cur_layer = tf.nn.relu(tf.matmul(x,weight)) + bias)
in_dimension = layer_dimension[i]
ses_loss = tf.reduce_mean(tf.square(y_ - cur_layer))#計算最終輸出與標準之間的loss
tf.add_to_collenction("losses",ses_loss)#把均方誤差也加入到集合裡
loss = tf.add_n(tf.get_collection("losses"))
#tf.get_collection返回一個列表,內容是這個集合的所有元素
#add_n()把輸入按照元素相加