[目錄]

tf.constant

tf.constant(value,dtype=None,shape=None,name=’Const’)
建立一個常量tensor，按照給出value來賦值，可以用shape來指定其形狀。value可以是一個數，也可以是一個list。
如果是一個數，那麼這個常亮中所有值的按該數來賦值。
如果是list,那麼len(value)一定要小於等於shape展開後的長度。賦值時，先將value中的值逐個存入。不夠的部分，則全部存入value的最後一個值。

a = tf.constant(2,shape=[2])
b = tf.constant(2
 
,shape=[2,2])
c = tf.constant([1,2,3],shape=[6])
d = tf.constant([1,2,3],shape=[3,2])

sess = tf.InteractiveSession()
print(sess.run(a))
#[2 2]
print(sess.run(b))
#[[2 2]
# [2 2]]
print(sess.run(c))
#[1 2 3 3 3 3]
print(sess.run(d))
#[[1 2]
# [3 3]
# [3 3]]

tf.random_normal | tf.truncated_normal | tf.random_uniform

tf.random_normal(shape,mean=0.0,stddev=1.0,dtype=tf.float32,seed=None,name=None)
tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)
tf.random_uniform(shape,minval=0,maxval=None,dtype=tf.float32,seed=None,name=None)
這幾個都是用於生成隨機數tensor的。尺寸是shape
random_normal: 正太分佈隨機數，均值mean,標準差stddev
truncated_normal:截斷正態分佈隨機數，均值mean,標準差stddev,不過只保留[mean-2*stddev,mean+2*stddev]範圍內的隨機數
random_uniform:均勻分佈隨機數，範圍為[minval,maxval]

sess = tf.InteractiveSession()
x = tf.random_normal(shape=[1,5],mean=0.0,stddev=1.0,dtype=tf.float32,seed=None,name=None)
print(sess.run(x))
#===>[[-0.36128798  0.58550537 -0.88363433 -0.2677258   1.05080092]]

tf.concat

tf.concat(concat_dim, values, name=’concat’)
tf.concat是連線兩個矩陣的操作,除去name引數用以指定該操作的name，與方法有關的一共兩個引數：
第一個引數concat_dim：必須是一個數，表明在哪一維上連線.

如果concat_dim是0，那麼在某一個shape的第一個維度上連，對應到實際，就是疊放到列上.

t1 = [[1, 2, 3], [4, 5, 6]]  
t2 = [[7, 8, 9], [10, 11, 12]]  
tf.concat(0, [t1, t2]) == > [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]  

如果concat_dim是1，那麼在某一個shape的第二個維度上連

t1 = [[1, 2, 3], [4, 5, 6]]  
t2 = [[7, 8, 9], [10, 11, 12]]  
tf.concat(1, [t1, t2]) ==> [[1, 2, 3, 7, 8, 9], [4, 5, 6, 10, 11, 12]]

如果有更高維，最後連線的依然是指定那個維：
values[i].shape = [D0, D1, … Dconcat_dim(i), …Dn]連線後就是：[D0, D1, … Rconcat_dim, …Dn]
第二個引數values：就是兩個或者一組待連線的tensor了

這裡要注意的是：如果是兩個向量，它們是無法呼叫tf.concat(1, [t1, t2]) 來連線的，因為它們對應的shape只有一個維度，當然不能在第二維上連了，雖然實際中兩個向量可以在行上連，但是放在程式裡是會報錯的
如果要連，必須要呼叫tf.expand_dims來擴維：

t1=tf.constant([1,2,3])  
t2=tf.constant([4,5,6])  
#concated = tf.concat(1, [t1,t2])這樣會報錯  
t1=tf.expand_dims(tf.constant([1,2,3]),1)  
t2=tf.expand_dims(tf.constant([4,5,6]),1)  
concated = tf.concat(1, [t1,t2])#這樣就是正確的  

tf.cast

cast(x, dtype, name=None)
將x的資料格式轉化成dtype.例如，原來x的資料格式是bool，那麼將其轉化成float以後，就能夠將其轉化成0和1的序列。反之也可以

a = tf.Variable([1,0,0,1,1])
b = tf.cast(a,dtype=tf.bool)
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
print(sess.run(b))
#[ True False False  True  True]

tf.multinomial

tf.multinomial(logits, num_samples, seed=None, name=None)
Args:

• logits: 2-D Tensor with shape [batch_size, num_classes]. Each slice [i, :] represents the unnormalized log probabilities for all classes.
• num_samples: 0-D. Number of independent samples to draw for each row slice.
• seed: A Python integer. Used to create a random seed for the distribution. See tf.set_random_seed for behavior.
• name: Optional name for the operation.

tf.squeeze()

squeeze(input, axis=None, name=None, squeeze_dims=None)
squeeze函式用於刪除張量中的單一維(Removes dimensions of size 1 from the shape of a tensor）
Args:
–input: A Tensor. The input to squeeze.
–axis: An optional list of ints. Defaults to [].If specified, only squeezes the dimensions listed. The dimension index starts at 0. It is an error to squeeze a dimension that is not 1.
– name: A name for the operation (optional).
– squeeze_dims: Deprecated keyword argument that is now axis.

Returns:
A Tensor. Has the same type as input.
Contains the same data as input, but has one or more dimensions of size 1 removed.
prettyprint
# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]
shape(squeeze(t)) ==> [2, 3]

prettyprint
# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]
shape(squeeze(t, [2, 4])) ==> [1, 2, 3, 1]


tf.random_normal | tf.truncated_normal | tf.random_uniform

tf.random_normal(shape,mean=0.0,stddev=1.0,dtype=tf.float32,seed=None,name=None)
tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)
tf.random_uniform(shape,minval=0,maxval=None,dtype=tf.float32,seed=None,name=None)
這幾個都是用於生成隨機數tensor的。尺寸是shape
random_normal: 正太分佈隨機數，均值mean,標準差stddev
truncated_normal:截斷正態分佈隨機數，均值mean,標準差stddev,不過只保留[mean-2*stddev,mean+2*stddev]範圍內的隨機數
random_uniform:均勻分佈隨機數，範圍為[minval,maxval]

tf.one_hot

a = tf.one_hot(indices, depth=10, on_value=None, off_value=None, axis=None, dtype=None, name=None)
Args:

• indices: A Tensor of indices.
• depth: A scalar defining the depth of the one hot dimension.
• on_value: A scalar defining the value to fill in output when indices[j] = i. (default: 1)
• off_value: A scalar defining the value to fill in output when indices[j] != i. (default: 0)
• axis: The axis to fill (default: -1, a new inner-most axis).
• dtype: The data type of the output tensor.

Returns:
output: The one-hot tensor.
examples:

indices = [0, 2, -1, 1]
  depth = 3
  on_value = 5.0
  off_value = 0.0
  axis = -1
  ```
  輸出是[4 *3]
 ```
  output =
  [5.0 0.0 0.0]  // one_hot(0)
  [0.0 0.0 5.0]  // one_hot(2)
  [0.0 0.0 0.0]  // one_hot(-1)

例2:


indices = [[0, 2], [1, -1]]
depth = 3
on_value = 1.0
off_value = 0.0
axis = -1

輸出是[2*2*3]

output =
[
[1.0, 0.0, 0.0] // one_hot(0)
[0.0, 0.0, 1.0] // one_hot(2)
][
[0.0, 1.0, 0.0] // one_hot(1)
[0.0, 0.0, 0.0] // one_hot(-1)
]


slim.fully_connected()

fully_connected(inputs,
num_outputs,
activation_fn=nn.relu,
normalizer_fn=None,
normalizer_params=None,
weights_initializer=initializers.xavier_initializer(),
weights_regularizer=None,
biases_initializer=init_ops.zeros_initializer(),
biases_regularizer=None,
reuse=None,
variables_collections=None,
outputs_collections=None,
trainable=True,
scope=None)

函式產生一個名為weights的變數，代表一個全連結權重矩陣，與輸入相乘生成一個隱層單元張量。若提供了’normalizer_fn’,則應用相關的正則化函式（如batch_norm）.另外，如果提供了‘biases_initializer’而沒有提供‘normalizer_fn’,那麼將生成‘biases’變數並加到隱層單元中去。最後，如果’activation_fn’不是‘None’，也會應用到隱層單元中去。
Args:
- inputs: A tensor of with at least rank 2 and value for the last dimension, i.e. [batch_size, depth], [None, None, None, channels].
- num_outputs: Integer or long, the number of output units in the layer.
- activation_fn: activation function, set to None to skip it and maintain a linear activation.
- normalizer_fn: normalization function to use instead of biases. If normalizer_fn is provided then biases_initializer andbiases_regularizer are ignored and biases are not created nor added.default set to None for no normalizer function
- normalizer_params: normalization function parameters.
- weights_initializer: An initializer for the weights.
- weights_regularizer: Optional regularizer for the weights.
- biases_initializer: An initializer for the biases. If None skip biases.
- biases_regularizer: Optional regularizer for the biases
- reuse: whether or not the layer and its variables should be reused. To beable to reuse the layer scope must be given.
- variables_collections: Optional list of collections for all the variables or a dictionary containing a different list of collections per variable.
- outputs_collections: collection to add the outputs.
- t**rainable:** If True also add variables to the graph collection GraphKeys.TRAINABLE_VARIABLES (see tf.Variable).
- scope: Optional scope for variable_scope.

slim.conv2d_transpose()

convolution2d_transpose(
inputs,
num_outputs,
kernel_size,
stride=1,
padding=’SAME’,
data_format=DATA_FORMAT_NHWC,
activation_fn=nn.relu,
normalizer_fn=None,
normalizer_params=None,
weights_initializer=initializers.xavier_initializer(),
weights_regularizer=None,
biases_initializer=init_ops.zeros_initializer(),
biases_regularizer=None,
reuse=None,
variables_collections=None,
outputs_collections=None,
trainable=True,
scope=None)
The function creates a variable called weights, representing the kernel, that is convolved with the input.
Args:
inputs: A 4-D Tensor of type float and shape [batch, height, width, in_channels] for NHWC data format or[batch, in_channels, height, width] for NCHW data format.
num_outputs: integer, the number of output filters.
kernel_size: a list of length 2 holding the [kernel_height, kernel_width] of the filters. Can be an int if both values are the same.
stride: a list of length 2: [stride_height, stride_width].Can be an int if both strides are the same. Note that presently both strides must have the same value.
padding: one of ‘VALID’ or ‘SAME’.
data_format: A string. NHWC (default) and NCHW are supported.
activation_fn: activation function, set to None to skip it and maintain a linear activation.
normalizer_fn: normalization function to use instead of biases. If normalizer_fn is provided then biases_initializer andbiases_regularizer are ignored and biases are not created nor added. default set to None for no normalizer function
normalizer_params: normalization function parameters.
weights_initializer: An initializer for the weights.
weights_regularizer: Optional regularizer for the weights.
biases_initializer: An initializer for the biases. If None skip biases.
biases_regularizer: Optional regularizer for the biases.
reuse: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.
variables_collections: optional list of collections for all the variables or a dictionary containing a different list of collection per variable.
outputs_collections: collection to add the outputs.
trainable: whether or not the variables should be trainable or not.
scope: Optional scope for variable_scope.

slim.flatten(inputs,outputs_collections=None,scope=None)

(注：import tensorflow.contrib.slim as slim)
將輸入扁平化但保留batch_size,假設第一維是batch。
Args:
inputs: a tensor of size [batch_size, …].
outputs_collections: collection to add the outputs.
scope: Optional scope for name_scope.
如：

test=([[[1,2,3],[4,5,6],[7,8,9]],[[10,11,12],[13,14,15],[16,17,27]],[[18,19,20],[21,22,23],[24,25,26]]])    #shape is (3,3,3)
test=slim.fatten(test)
test.eval()
array([[ 1,  2,  3, ...,  7,  8,  9],
       [10, 11, 12, ..., 16, 17, 27],
       [18, 19, 20, ..., 24, 25, 26]], dtype=int32)
test.get_shape()
TensorShape([Dimension(3), Dimension(9)])  #(3,9)

tf.train.AdamOptimizer()

Adam，這是一種基於一階梯度來優化隨機目標函式的演算法。隨即目標函式的含義是，在訓練過程的每一次迭代中，目標函式是不一樣的。有時候因為記憶體不夠大或者其他的原因，演算法不會一下子讀取全部記錄來計算誤差，而是選擇選擇對資料集進行分割，在每次迭代中只讀取一部分記錄進行訓練，這一部分記錄稱為minibatch，這樣每次迭代所使用的小批量資料集就是不同的，資料集不同，損失函式就不同，因此就有隨機目標函式的說法。另外還有一個原因就是，採用小批量方式來進行訓練，可以降低收斂到區域性最優的風險.
Adam 這個名字來源於 adaptive moment estimation，自適應矩估計。概率論中矩的含義是：如果一個隨機變數 X 服從某個分佈，X 的一階矩是 E(X)，也就是樣本平均值，X 的二階矩就是 E(X^2)，也就是樣本平方的平均值。Adam 演算法根據損失函式對每個引數的梯度的一階矩估計和二階矩估計動態調整針對於每個引數的學習速率。Adam 也是基於梯度下降的方法，但是每次迭代引數的學習步長都有一個確定的範圍，不會因為很大的梯度導致很大的學習步長，引數的值比較穩定。

初始化Adam優化器：
tf.train.AdamOptimizer.init(learning_rate=0.001,beta1=0.9, beta2=0.999, epsilon=1e-08,use_locking=False, name=’Adam’)

待續