init( decay, num_updates=None, zero_debias=False, name='ExponentialMovingAverage' )

The moving averages are computed using exponential decay. You specify the decay value when creating the ExponentialMovingAverage object. The shadow variables are initialized with the same initial values as the trained variables. When you run the ops to maintain the moving averages, each shadow variable is updated with the formula:

shadow_variable = decay * shadow_variable + (1 - decay) * variable

The optional num_updates parameter allows one to tweak the decay rate dynamically. It is typical to pass the count of training steps, usually kept in a variable that is incremented at each step,in which case the decay rate is lower at the start of training.This makes moving averages move faster. If passed, the actual decay rate used is:

min(decay, (1 + num_updates) / (10 + num_updates))

import os
import tensorflow as tf

os.environ["CUDA_VISIBLE_DEVICES"] = "1"

# The variables and Tensors must be of types float.
v1 = tf.Variable(0, dtype=tf.float32)

# shadow variables are created with trainable=False
# and added to the GraphKeys.ALL_VARIABLES collection.
num_updates = tf.Variable(0, trainable=False)

ema = tf.train.ExponentialMovingAverage(0.99, num_updates)


maintain_averages_op = ema.apply([v1])

with tf.Session() as sess:
    init_op = tf.global_variables_initializer()
    sess.run(init_op)
    sess.run(tf.assign(v1, 5))
    for i in range(0, 10, 1):
        sess.run(maintain_averages_op)
        print(sess.run([v1, ema.average(v1)]))

輸出如下：

[5.0, 4.5] [5.0, 4.95] [5.0, 4.995] [5.0, 4.9995] [5.0, 4.99995] [5.0, 4.999995] [5.0, 4.9999995] [5.0, 5.0] [5.0, 5.0] [5.0, 5.0]

分析：由於第一次，我們的num_updates為0，故min(decay, (1 + num_updates) / (10 + num_updates))取最小值為1/10=0.1，因此第一次計算得：0*0.1+5*(1-0.1)=4.5，以此類推～

提示：影子變數的初值應該是0