2. 程式人生 > 實用技巧 >RL實踐1——值迭代求解隨機策略

RL實踐1——值迭代求解隨機策略

阿新 • • 發佈：2020-09-02

RL實踐1——值迭代求解隨機策略

參考自葉強《強化學習》第三講，方格世界—— 使用 動態規劃 求解隨機策略

動態規劃的使用條件時MDP已知，在簡單遊戲中，這個條件時顯然成立的
使用Value iteration的方法求解每個狀態的價值函式，迭代收斂之後，對應最優策略生成。

問題定義


從方格狀態走到終止狀態（灰色標記）

Python程式碼及註釋

值得注意的是，知乎原版的註釋是錯誤的，採用的是同步更新

有三個trick可以加快運算速度（對於大規模問題）

  • in-place DP：新值直接替換舊值，只儲存一個v(s)，
    • 非同步更新，提高效率
    • 缺點：更新順序影響收斂性
  • Prioritised sweeping：state的影響力排序
    • 比較貝爾曼誤差絕對值，大的更新，小的忽略
  • Real-time DP：遍歷過的才更新
    • 省去了agent 未遍歷的狀態s，對於稀疏任務效率提升極大
# 狀態集合
states = [i for i in range(16)]
# 價值集合
values = [0 for _ in range(16)]
# 動作集：
actions = ["n", "e", "s", "w"]
# 動作字典：
ds_actions = {"n": -4, "e": 1, "s": 4, "w": -1}
# 衰減率
gamma = 1.00


# 定義MDP
def nextState(s, a):
    next_state = s
    if (s%4 == 0 and a == "w") or (s<4 and a == "n") or \
      ((s+1)%4 == 0 and a == "e") or (s > 11 and a == "s"):
        pass
    else:
        ds = ds_actions[a]
        next_state = s + ds
    return next_state


# 定義獎勵
def rewardOf(s):
    return 0 if s in [0, 15] else -1


# 判斷是否結束
def isTerminateState(s):
    return s in [0, 15]


# 獲取所有可能的next state 集合
def getSuccessors(s):
    successors = []
    if isTerminateState(s):
        return successors
    for a in actions:
        next_state = nextState(s, a)
        # if s != next_state:
        successors.append(next_state)
    return successors


# 更新當前位置的價值函式
def updateValue(s):
    sucessors = getSuccessors(s)
    newValue = 0  # values[s]
    num = 4  # len(successors)
    reward = rewardOf(s)
    for next_state in sucessors:
        newValue += 1.00 / num * (reward + gamma * values[next_state])
    return newValue


# 列印所有狀態對應價值函式
def printValue(v):
    for i in range(16):
        print('{0:>6.2f}'.format(v[i]), end=" ")
        if (i + 1) % 4 == 0:
            print("")
    print()


# 一次迭代
# 這裡採用的是同步更新，不是非同步更新。建立了newvalues陣列，遍歷過states後，統一更新global values
def performOneIteration():
    newValues = [0 for _ in range(16)]
    for s in states:
        newValues[s] = updateValue(s)
    global values
    values = newValues
    printValue(values)

 
# 主函式
def main():
    max_iterate_times = 160
    cur_iterate_times = 0
    while cur_iterate_times <= max_iterate_times:
        print("Iterate No.{0}".format(cur_iterate_times))
        performOneIteration()
        cur_iterate_times += 1
    printValue(values)

if __name__ == '__main__':
    main()

運算結果如下

Iterate No.0
  0.00  -1.00  -1.00  -1.00 
 -1.00  -1.00  -1.00  -1.00 
 -1.00  -1.00  -1.00  -1.00 
 -1.00  -1.00  -1.00   0.00 

Iterate No.1
  0.00  -1.75  -2.00  -2.00 
 -1.75  -2.00  -2.00  -2.00 
 -2.00  -2.00  -2.00  -1.75 
 -2.00  -2.00  -1.75   0.00 

Iterate No.2
  0.00  -2.44  -2.94  -3.00 
 -2.44  -2.88  -3.00  -2.94 
 -2.94  -3.00  -2.88  -2.44 
 -3.00  -2.94  -2.44   0.00 

Iterate No.3
  0.00  -3.06  -3.84  -3.97 
 -3.06  -3.72  -3.91  -3.84 
 -3.84  -3.91  -3.72  -3.06 
 -3.97  -3.84  -3.06   0.00 

Iterate No.4
  0.00  -3.66  -4.70  -4.91 
 -3.66  -4.48  -4.78  -4.70 
 -4.70  -4.78  -4.48  -3.66 
 -4.91  -4.70  -3.66   0.00 

Iterate No.5
  0.00  -4.21  -5.51  -5.80 
 -4.21  -5.22  -5.59  -5.51 
 -5.51  -5.59  -5.22  -4.21 
 -5.80  -5.51  -4.21   0.00 

Iterate No.6
  0.00  -4.73  -6.28  -6.66 
 -4.73  -5.90  -6.36  -6.28 
 -6.28  -6.36  -5.90  -4.73 
 -6.66  -6.28  -4.73   0.00 

Iterate No.7
  0.00  -5.23  -7.01  -7.47 
 -5.23  -6.55  -7.09  -7.01 
 -7.01  -7.09  -6.55  -5.23 
 -7.47  -7.01  -5.23   0.00 

Iterate No.8
  0.00  -5.70  -7.70  -8.24 
 -5.70  -7.16  -7.78  -7.70 
 -7.70  -7.78  -7.16  -5.70 
 -8.24  -7.70  -5.70   0.00 

Iterate No.9
  0.00  -6.14  -8.35  -8.97 
 -6.14  -7.74  -8.43  -8.35 
 -8.35  -8.43  -7.74  -6.14 
 -8.97  -8.35  -6.14   0.00 

Iterate No.10
  0.00  -6.56  -8.97  -9.66 
 -6.56  -8.28  -9.04  -8.97 
 -8.97  -9.04  -8.28  -6.56 
 -9.66  -8.97  -6.56   0.00 

Iterate No.11
  0.00  -6.95  -9.56 -10.32 
 -6.95  -8.80  -9.63  -9.56 
 -9.56  -9.63  -8.80  -6.95 
-10.32  -9.56  -6.95   0.00 

Iterate No.12
  0.00  -7.33 -10.11 -10.94 
 -7.33  -9.29 -10.18 -10.11 
-10.11 -10.18  -9.29  -7.33 
-10.94 -10.11  -7.33   0.00 

Iterate No.13
  0.00  -7.68 -10.64 -11.53 
 -7.68  -9.75 -10.70 -10.64 
-10.64 -10.70  -9.75  -7.68 
-11.53 -10.64  -7.68   0.00 

Iterate No.14
  0.00  -8.02 -11.14 -12.08 
 -8.02 -10.19 -11.20 -11.14 
-11.14 -11.20 -10.19  -8.02 
-12.08 -11.14  -8.02   0.00 

Iterate No.15
  0.00  -8.34 -11.61 -12.61 
 -8.34 -10.61 -11.66 -11.61 
-11.61 -11.66 -10.61  -8.34 
-12.61 -11.61  -8.34   0.00 

Iterate No.16
  0.00  -8.64 -12.06 -13.11 
 -8.64 -11.00 -12.11 -12.06 
-12.06 -12.11 -11.00  -8.64 
-13.11 -12.06  -8.64   0.00 

Iterate No.17
  0.00  -8.92 -12.48 -13.58 
 -8.92 -11.37 -12.53 -12.48 
-12.48 -12.53 -11.37  -8.92 
-13.58 -12.48  -8.92   0.00 

Iterate No.18
  0.00  -9.19 -12.88 -14.03 
 -9.19 -11.73 -12.93 -12.88 
-12.88 -12.93 -11.73  -9.19 
-14.03 -12.88  -9.19   0.00 

Iterate No.19
  0.00  -9.45 -13.26 -14.45 
 -9.45 -12.06 -13.30 -13.26 
-13.26 -13.30 -12.06  -9.45 
-14.45 -13.26  -9.45   0.00 

Iterate No.20
  0.00  -9.69 -13.62 -14.86 
 -9.69 -12.38 -13.66 -13.62 
-13.62 -13.66 -12.38  -9.69 
-14.86 -13.62  -9.69   0.00 

Iterate No.21
  0.00  -9.92 -13.96 -15.24 
 -9.92 -12.67 -14.00 -13.96 
-13.96 -14.00 -12.67  -9.92 
-15.24 -13.96  -9.92   0.00 

Iterate No.22
  0.00 -10.14 -14.28 -15.60 
-10.14 -12.96 -14.32 -14.28 
-14.28 -14.32 -12.96 -10.14 
-15.60 -14.28 -10.14   0.00 

Iterate No.23
  0.00 -10.34 -14.58 -15.94 
-10.34 -13.23 -14.62 -14.58 
-14.58 -14.62 -13.23 -10.34 
-15.94 -14.58 -10.34   0.00 

Iterate No.24
  0.00 -10.54 -14.87 -16.26 
-10.54 -13.48 -14.90 -14.87 
-14.87 -14.90 -13.48 -10.54 
-16.26 -14.87 -10.54   0.00 

Iterate No.25
  0.00 -10.72 -15.14 -16.56 
-10.72 -13.72 -15.17 -15.14 
-15.14 -15.17 -13.72 -10.72 
-16.56 -15.14 -10.72   0.00 

Iterate No.26
  0.00 -10.90 -15.40 -16.85 
-10.90 -13.95 -15.43 -15.40 
-15.40 -15.43 -13.95 -10.90 
-16.85 -15.40 -10.90   0.00 

Iterate No.27
  0.00 -11.06 -15.65 -17.13 
-11.06 -14.16 -15.67 -15.65 
-15.65 -15.67 -14.16 -11.06 
-17.13 -15.65 -11.06   0.00 

Iterate No.28
  0.00 -11.22 -15.88 -17.39 
-11.22 -14.37 -15.90 -15.88 
-15.88 -15.90 -14.37 -11.22 
-17.39 -15.88 -11.22   0.00 

Iterate No.29
  0.00 -11.37 -16.10 -17.63 
-11.37 -14.56 -16.12 -16.10 
-16.10 -16.12 -14.56 -11.37 
-17.63 -16.10 -11.37   0.00 

Iterate No.30
  0.00 -11.51 -16.30 -17.86 
-11.51 -14.74 -16.33 -16.30 
-16.30 -16.33 -14.74 -11.51 
-17.86 -16.30 -11.51   0.00 

Iterate No.31
  0.00 -11.64 -16.50 -18.08 
-11.64 -14.92 -16.52 -16.50 
-16.50 -16.52 -14.92 -11.64 
-18.08 -16.50 -11.64   0.00 

Iterate No.32
  0.00 -11.76 -16.69 -18.29 
-11.76 -15.08 -16.71 -16.69 
-16.69 -16.71 -15.08 -11.76 
-18.29 -16.69 -11.76   0.00 

Iterate No.33
  0.00 -11.88 -16.86 -18.49 
-11.88 -15.24 -16.88 -16.86 
-16.86 -16.88 -15.24 -11.88 
-18.49 -16.86 -11.88   0.00 

Iterate No.34
  0.00 -12.00 -17.03 -18.68 
-12.00 -15.38 -17.05 -17.03 
-17.03 -17.05 -15.38 -12.00 
-18.68 -17.03 -12.00   0.00 

Iterate No.35
  0.00 -12.10 -17.19 -18.85 
-12.10 -15.52 -17.21 -17.19 
-17.19 -17.21 -15.52 -12.10 
-18.85 -17.19 -12.10   0.00 

Iterate No.36
  0.00 -12.20 -17.34 -19.02 
-12.20 -15.65 -17.36 -17.34 
-17.34 -17.36 -15.65 -12.20 
-19.02 -17.34 -12.20   0.00 

Iterate No.37
  0.00 -12.30 -17.48 -19.18 
-12.30 -15.78 -17.50 -17.48 
-17.48 -17.50 -15.78 -12.30 
-19.18 -17.48 -12.30   0.00 

Iterate No.38
  0.00 -12.39 -17.61 -19.33 
-12.39 -15.90 -17.63 -17.61 
-17.61 -17.63 -15.90 -12.39 
-19.33 -17.61 -12.39   0.00 

Iterate No.39
  0.00 -12.47 -17.74 -19.47 
-12.47 -16.01 -17.76 -17.74 
-17.74 -17.76 -16.01 -12.47 
-19.47 -17.74 -12.47   0.00 

Iterate No.40
  0.00 -12.56 -17.86 -19.61 
-12.56 -16.12 -17.87 -17.86 
-17.86 -17.87 -16.12 -12.56 
-19.61 -17.86 -12.56   0.00 

Iterate No.41
  0.00 -12.63 -17.97 -19.73 
-12.63 -16.22 -17.99 -17.97 
-17.97 -17.99 -16.22 -12.63 
-19.73 -17.97 -12.63   0.00 

Iterate No.42
  0.00 -12.71 -18.08 -19.85 
-12.71 -16.31 -18.09 -18.08 
-18.08 -18.09 -16.31 -12.71 
-19.85 -18.08 -12.71   0.00 

Iterate No.43
  0.00 -12.77 -18.18 -19.97 
-12.77 -16.40 -18.20 -18.18 
-18.18 -18.20 -16.40 -12.77 
-19.97 -18.18 -12.77   0.00 

Iterate No.44
  0.00 -12.84 -18.28 -20.08 
-12.84 -16.49 -18.29 -18.28 
-18.28 -18.29 -16.49 -12.84 
-20.08 -18.28 -12.84   0.00 

Iterate No.45
  0.00 -12.90 -18.37 -20.18 
-12.90 -16.57 -18.38 -18.37 
-18.37 -18.38 -16.57 -12.90 
-20.18 -18.37 -12.90   0.00 

Iterate No.46
  0.00 -12.96 -18.46 -20.28 
-12.96 -16.64 -18.47 -18.46 
-18.46 -18.47 -16.64 -12.96 
-20.28 -18.46 -12.96   0.00 

Iterate No.47
  0.00 -13.02 -18.54 -20.37 
-13.02 -16.71 -18.55 -18.54 
-18.54 -18.55 -16.71 -13.02 
-20.37 -18.54 -13.02   0.00 

Iterate No.48
  0.00 -13.07 -18.62 -20.45 
-13.07 -16.78 -18.63 -18.62 
-18.62 -18.63 -16.78 -13.07 
-20.45 -18.62 -13.07   0.00 

Iterate No.49
  0.00 -13.12 -18.69 -20.54 
-13.12 -16.85 -18.70 -18.69 
-18.69 -18.70 -16.85 -13.12 
-20.54 -18.69 -13.12   0.00 

Iterate No.50
  0.00 -13.16 -18.76 -20.61 
-13.16 -16.91 -18.77 -18.76 
-18.76 -18.77 -16.91 -13.16 
-20.61 -18.76 -13.16   0.00 

Iterate No.51
  0.00 -13.21 -18.83 -20.69 
-13.21 -16.97 -18.84 -18.83 
-18.83 -18.84 -16.97 -13.21 
-20.69 -18.83 -13.21   0.00 

Iterate No.52
  0.00 -13.25 -18.89 -20.76 
-13.25 -17.02 -18.90 -18.89 
-18.89 -18.90 -17.02 -13.25 
-20.76 -18.89 -13.25   0.00 

Iterate No.53
  0.00 -13.29 -18.95 -20.82 
-13.29 -17.07 -18.96 -18.95 
-18.95 -18.96 -17.07 -13.29 
-20.82 -18.95 -13.29   0.00 

Iterate No.54
  0.00 -13.33 -19.00 -20.89 
-13.33 -17.12 -19.01 -19.00 
-19.00 -19.01 -17.12 -13.33 
-20.89 -19.00 -13.33   0.00 

Iterate No.55
  0.00 -13.36 -19.06 -20.95 
-13.36 -17.17 -19.06 -19.06 
-19.06 -19.06 -17.17 -13.36 
-20.95 -19.06 -13.36   0.00 

Iterate No.56
  0.00 -13.40 -19.11 -21.00 
-13.40 -17.21 -19.11 -19.11 
-19.11 -19.11 -17.21 -13.40 
-21.00 -19.11 -13.40   0.00 

Iterate No.57
  0.00 -13.43 -19.16 -21.05 
-13.43 -17.26 -19.16 -19.16 
-19.16 -19.16 -17.26 -13.43 
-21.05 -19.16 -13.43   0.00 

Iterate No.58
  0.00 -13.46 -19.20 -21.10 
-13.46 -17.30 -19.21 -19.20 
-19.20 -19.21 -17.30 -13.46 
-21.10 -19.20 -13.46   0.00 

Iterate No.59
  0.00 -13.49 -19.24 -21.15 
-13.49 -17.33 -19.25 -19.24 
-19.24 -19.25 -17.33 -13.49 
-21.15 -19.24 -13.49   0.00 

Iterate No.60
  0.00 -13.52 -19.28 -21.20 
-13.52 -17.37 -19.29 -19.28 
-19.28 -19.29 -17.37 -13.52 
-21.20 -19.28 -13.52   0.00 

Iterate No.61
  0.00 -13.54 -19.32 -21.24 
-13.54 -17.40 -19.33 -19.32 
-19.32 -19.33 -17.40 -13.54 
-21.24 -19.32 -13.54   0.00 

Iterate No.62
  0.00 -13.57 -19.36 -21.28 
-13.57 -17.43 -19.36 -19.36 
-19.36 -19.36 -17.43 -13.57 
-21.28 -19.36 -13.57   0.00 

Iterate No.63
  0.00 -13.59 -19.39 -21.32 
-13.59 -17.46 -19.40 -19.39 
-19.39 -19.40 -17.46 -13.59 
-21.32 -19.39 -13.59   0.00 

Iterate No.64
  0.00 -13.61 -19.42 -21.36 
-13.61 -17.49 -19.43 -19.42 
-19.42 -19.43 -17.49 -13.61 
-21.36 -19.42 -13.61   0.00 

Iterate No.65
  0.00 -13.63 -19.45 -21.39 
-13.63 -17.52 -19.46 -19.45 
-19.45 -19.46 -17.52 -13.63 
-21.39 -19.45 -13.63   0.00 

Iterate No.66
  0.00 -13.65 -19.48 -21.42 
-13.65 -17.54 -19.49 -19.48 
-19.48 -19.49 -17.54 -13.65 
-21.42 -19.48 -13.65   0.00 

Iterate No.67
  0.00 -13.67 -19.51 -21.45 
-13.67 -17.57 -19.51 -19.51 
-19.51 -19.51 -17.57 -13.67 
-21.45 -19.51 -13.67   0.00 

Iterate No.68
  0.00 -13.69 -19.54 -21.48 
-13.69 -17.59 -19.54 -19.54 
-19.54 -19.54 -17.59 -13.69 
-21.48 -19.54 -13.69   0.00 

Iterate No.69
  0.00 -13.70 -19.56 -21.51 
-13.70 -17.61 -19.56 -19.56 
-19.56 -19.56 -17.61 -13.70 
-21.51 -19.56 -13.70   0.00 

Iterate No.70
  0.00 -13.72 -19.58 -21.54 
-13.72 -17.63 -19.59 -19.58 
-19.58 -19.59 -17.63 -13.72 
-21.54 -19.58 -13.72   0.00 

Iterate No.71
  0.00 -13.73 -19.61 -21.56 
-13.73 -17.65 -19.61 -19.61 
-19.61 -19.61 -17.65 -13.73 
-21.56 -19.61 -13.73   0.00 

Iterate No.72
  0.00 -13.75 -19.63 -21.58 
-13.75 -17.67 -19.63 -19.63 
-19.63 -19.63 -17.67 -13.75 
-21.58 -19.63 -13.75   0.00 

Iterate No.73
  0.00 -13.76 -19.65 -21.61 
-13.76 -17.69 -19.65 -19.65 
-19.65 -19.65 -17.69 -13.76 
-21.61 -19.65 -13.76   0.00 

Iterate No.74
  0.00 -13.77 -19.67 -21.63 
-13.77 -17.71 -19.67 -19.67 
-19.67 -19.67 -17.71 -13.77 
-21.63 -19.67 -13.77   0.00 

Iterate No.75
  0.00 -13.79 -19.68 -21.65 
-13.79 -17.72 -19.69 -19.68 
-19.68 -19.69 -17.72 -13.79 
-21.65 -19.68 -13.79   0.00 

Iterate No.76
  0.00 -13.80 -19.70 -21.67 
-13.80 -17.74 -19.70 -19.70 
-19.70 -19.70 -17.74 -13.80 
-21.67 -19.70 -13.80   0.00 

Iterate No.77
  0.00 -13.81 -19.72 -21.68 
-13.81 -17.75 -19.72 -19.72 
-19.72 -19.72 -17.75 -13.81 
-21.68 -19.72 -13.81   0.00 

Iterate No.78
  0.00 -13.82 -19.73 -21.70 
-13.82 -17.76 -19.73 -19.73 
-19.73 -19.73 -17.76 -13.82 
-21.70 -19.73 -13.82   0.00 

Iterate No.79
  0.00 -13.83 -19.75 -21.72 
-13.83 -17.78 -19.75 -19.75 
-19.75 -19.75 -17.78 -13.83 
-21.72 -19.75 -13.83   0.00 

Iterate No.80
  0.00 -13.84 -19.76 -21.73 
-13.84 -17.79 -19.76 -19.76 
-19.76 -19.76 -17.79 -13.84 
-21.73 -19.76 -13.84   0.00 

Iterate No.81
  0.00 -13.85 -19.77 -21.75 
-13.85 -17.80 -19.77 -19.77 
-19.77 -19.77 -17.80 -13.85 
-21.75 -19.77 -13.85   0.00 

Iterate No.82
  0.00 -13.85 -19.78 -21.76 
-13.85 -17.81 -19.79 -19.78 
-19.78 -19.79 -17.81 -13.85 
-21.76 -19.78 -13.85   0.00 

Iterate No.83
  0.00 -13.86 -19.80 -21.77 
-13.86 -17.82 -19.80 -19.80 
-19.80 -19.80 -17.82 -13.86 
-21.77 -19.80 -13.86   0.00 

Iterate No.84
  0.00 -13.87 -19.81 -21.78 
-13.87 -17.83 -19.81 -19.81 
-19.81 -19.81 -17.83 -13.87 
-21.78 -19.81 -13.87   0.00 

Iterate No.85
  0.00 -13.88 -19.82 -21.80 
-13.88 -17.84 -19.82 -19.82 
-19.82 -19.82 -17.84 -13.88 
-21.80 -19.82 -13.88   0.00 

Iterate No.86
  0.00 -13.88 -19.83 -21.81 
-13.88 -17.85 -19.83 -19.83 
-19.83 -19.83 -17.85 -13.88 
-21.81 -19.83 -13.88   0.00 

Iterate No.87
  0.00 -13.89 -19.84 -21.82 
-13.89 -17.86 -19.84 -19.84 
-19.84 -19.84 -17.86 -13.89 
-21.82 -19.84 -13.89   0.00 

Iterate No.88
  0.00 -13.90 -19.84 -21.83 
-13.90 -17.86 -19.85 -19.84 
-19.84 -19.85 -17.86 -13.90 
-21.83 -19.84 -13.90   0.00 

Iterate No.89
  0.00 -13.90 -19.85 -21.84 
-13.90 -17.87 -19.85 -19.85 
-19.85 -19.85 -17.87 -13.90 
-21.84 -19.85 -13.90   0.00 

Iterate No.90
  0.00 -13.91 -19.86 -21.84 
-13.91 -17.88 -19.86 -19.86 
-19.86 -19.86 -17.88 -13.91 
-21.84 -19.86 -13.91   0.00 

Iterate No.91
  0.00 -13.91 -19.87 -21.85 
-13.91 -17.88 -19.87 -19.87 
-19.87 -19.87 -17.88 -13.91 
-21.85 -19.87 -13.91   0.00 

Iterate No.92
  0.00 -13.92 -19.88 -21.86 
-13.92 -17.89 -19.88 -19.88 
-19.88 -19.88 -17.89 -13.92 
-21.86 -19.88 -13.92   0.00 

Iterate No.93
  0.00 -13.92 -19.88 -21.87 
-13.92 -17.90 -19.88 -19.88 
-19.88 -19.88 -17.90 -13.92 
-21.87 -19.88 -13.92   0.00 

Iterate No.94
  0.00 -13.92 -19.89 -21.87 
-13.92 -17.90 -19.89 -19.89 
-19.89 -19.89 -17.90 -13.92 
-21.87 -19.89 -13.92   0.00 

Iterate No.95
  0.00 -13.93 -19.89 -21.88 
-13.93 -17.91 -19.89 -19.89 
-19.89 -19.89 -17.91 -13.93 
-21.88 -19.89 -13.93   0.00 

Iterate No.96
  0.00 -13.93 -19.90 -21.89 
-13.93 -17.91 -19.90 -19.90 
-19.90 -19.90 -17.91 -13.93 
-21.89 -19.90 -13.93   0.00 

Iterate No.97
  0.00 -13.94 -19.91 -21.89 
-13.94 -17.92 -19.91 -19.91 
-19.91 -19.91 -17.92 -13.94 
-21.89 -19.91 -13.94   0.00 

Iterate No.98
  0.00 -13.94 -19.91 -21.90 
-13.94 -17.92 -19.91 -19.91 
-19.91 -19.91 -17.92 -13.94 
-21.90 -19.91 -13.94   0.00 

Iterate No.99
  0.00 -13.94 -19.91 -21.90 
-13.94 -17.93 -19.92 -19.91 
-19.91 -19.92 -17.93 -13.94 
-21.90 -19.91 -13.94   0.00 

Iterate No.100
  0.00 -13.95 -19.92 -21.91 
-13.95 -17.93 -19.92 -19.92 
-19.92 -19.92 -17.93 -13.95 
-21.91 -19.92 -13.95   0.00 

Iterate No.101
  0.00 -13.95 -19.92 -21.91 
-13.95 -17.93 -19.92 -19.92 
-19.92 -19.92 -17.93 -13.95 
-21.91 -19.92 -13.95   0.00 

Iterate No.102
  0.00 -13.95 -19.93 -21.92 
-13.95 -17.94 -19.93 -19.93 
-19.93 -19.93 -17.94 -13.95 
-21.92 -19.93 -13.95   0.00 

Iterate No.103
  0.00 -13.95 -19.93 -21.92 
-13.95 -17.94 -19.93 -19.93 
-19.93 -19.93 -17.94 -13.95 
-21.92 -19.93 -13.95   0.00 

Iterate No.104
  0.00 -13.96 -19.94 -21.93 
-13.96 -17.94 -19.94 -19.94 
-19.94 -19.94 -17.94 -13.96 
-21.93 -19.94 -13.96   0.00 

Iterate No.105
  0.00 -13.96 -19.94 -21.93 
-13.96 -17.95 -19.94 -19.94 
-19.94 -19.94 -17.95 -13.96 
-21.93 -19.94 -13.96   0.00 

Iterate No.106
  0.00 -13.96 -19.94 -21.94 
-13.96 -17.95 -19.94 -19.94 
-19.94 -19.94 -17.95 -13.96 
-21.94 -19.94 -13.96   0.00 

Iterate No.107
  0.00 -13.96 -19.95 -21.94 
-13.96 -17.95 -19.95 -19.95 
-19.95 -19.95 -17.95 -13.96 
-21.94 -19.95 -13.96   0.00 

Iterate No.108
  0.00 -13.96 -19.95 -21.94 
-13.96 -17.95 -19.95 -19.95 
-19.95 -19.95 -17.95 -13.96 
-21.94 -19.95 -13.96   0.00 

Iterate No.109
  0.00 -13.97 -19.95 -21.94 
-13.97 -17.96 -19.95 -19.95 
-19.95 -19.95 -17.96 -13.97 
-21.94 -19.95 -13.97   0.00 

Iterate No.110
  0.00 -13.97 -19.95 -21.95 
-13.97 -17.96 -19.95 -19.95 
-19.95 -19.95 -17.96 -13.97 
-21.95 -19.95 -13.97   0.00 

Iterate No.111
  0.00 -13.97 -19.96 -21.95 
-13.97 -17.96 -19.96 -19.96 
-19.96 -19.96 -17.96 -13.97 
-21.95 -19.96 -13.97   0.00 

Iterate No.112
  0.00 -13.97 -19.96 -21.95 
-13.97 -17.96 -19.96 -19.96 
-19.96 -19.96 -17.96 -13.97 
-21.95 -19.96 -13.97   0.00 

Iterate No.113
  0.00 -13.97 -19.96 -21.96 
-13.97 -17.97 -19.96 -19.96 
-19.96 -19.96 -17.97 -13.97 
-21.96 -19.96 -13.97   0.00 

Iterate No.114
  0.00 -13.97 -19.96 -21.96 
-13.97 -17.97 -19.96 -19.96 
-19.96 -19.96 -17.97 -13.97 
-21.96 -19.96 -13.97   0.00 

Iterate No.115
  0.00 -13.98 -19.96 -21.96 
-13.98 -17.97 -19.96 -19.96 
-19.96 -19.96 -17.97 -13.98 
-21.96 -19.96 -13.98   0.00 

Iterate No.116
  0.00 -13.98 -19.97 -21.96 
-13.98 -17.97 -19.97 -19.97 
-19.97 -19.97 -17.97 -13.98 
-21.96 -19.97 -13.98   0.00 

Iterate No.117
  0.00 -13.98 -19.97 -21.96 
-13.98 -17.97 -19.97 -19.97 
-19.97 -19.97 -17.97 -13.98 
-21.96 -19.97 -13.98   0.00 

Iterate No.118
  0.00 -13.98 -19.97 -21.97 
-13.98 -17.97 -19.97 -19.97 
-19.97 -19.97 -17.97 -13.98 
-21.97 -19.97 -13.98   0.00 

Iterate No.119
  0.00 -13.98 -19.97 -21.97 
-13.98 -17.97 -19.97 -19.97 
-19.97 -19.97 -17.97 -13.98 
-21.97 -19.97 -13.98   0.00 

Iterate No.120
  0.00 -13.98 -19.97 -21.97 
-13.98 -17.98 -19.97 -19.97 
-19.97 -19.97 -17.98 -13.98 
-21.97 -19.97 -13.98   0.00 

Iterate No.121
  0.00 -13.98 -19.97 -21.97 
-13.98 -17.98 -19.97 -19.97 
-19.97 -19.97 -17.98 -13.98 
-21.97 -19.97 -13.98   0.00 

Iterate No.122
  0.00 -13.98 -19.98 -21.97 
-13.98 -17.98 -19.98 -19.98 
-19.98 -19.98 -17.98 -13.98 
-21.97 -19.98 -13.98   0.00 

Iterate No.123
  0.00 -13.98 -19.98 -21.97 
-13.98 -17.98 -19.98 -19.98 
-19.98 -19.98 -17.98 -13.98 
-21.97 -19.98 -13.98   0.00 

Iterate No.124
  0.00 -13.99 -19.98 -21.98 
-13.99 -17.98 -19.98 -19.98 
-19.98 -19.98 -17.98 -13.99 
-21.98 -19.98 -13.99   0.00 

Iterate No.125
  0.00 -13.99 -19.98 -21.98 
-13.99 -17.98 -19.98 -19.98 
-19.98 -19.98 -17.98 -13.99 
-21.98 -19.98 -13.99   0.00 

Iterate No.126
  0.00 -13.99 -19.98 -21.98 
-13.99 -17.98 -19.98 -19.98 
-19.98 -19.98 -17.98 -13.99 
-21.98 -19.98 -13.99   0.00 

Iterate No.127
  0.00 -13.99 -19.98 -21.98 
-13.99 -17.98 -19.98 -19.98 
-19.98 -19.98 -17.98 -13.99 
-21.98 -19.98 -13.99   0.00 

Iterate No.128
  0.00 -13.99 -19.98 -21.98 
-13.99 -17.98 -19.98 -19.98 
-19.98 -19.98 -17.98 -13.99 
-21.98 -19.98 -13.99   0.00 

Iterate No.129
  0.00 -13.99 -19.98 -21.98 
-13.99 -17.99 -19.98 -19.98 
-19.98 -19.98 -17.99 -13.99 
-21.98 -19.98 -13.99   0.00 

Iterate No.130
  0.00 -13.99 -19.98 -21.98 
-13.99 -17.99 -19.98 -19.98 
-19.98 -19.98 -17.99 -13.99 
-21.98 -19.98 -13.99   0.00 

Iterate No.131
  0.00 -13.99 -19.99 -21.98 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.98 -19.99 -13.99   0.00 

Iterate No.132
  0.00 -13.99 -19.99 -21.98 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.98 -19.99 -13.99   0.00 

Iterate No.133
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.134
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.135
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.136
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.137
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.138
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.139
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.140
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.141
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.142
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.143
  0.00 -13.99 -19.99 -21.99 
-13.99 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -13.99 
-21.99 -19.99 -13.99   0.00 

Iterate No.144
  0.00 -14.00 -19.99 -21.99 
-14.00 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -14.00 
-21.99 -19.99 -14.00   0.00 

Iterate No.145
  0.00 -14.00 -19.99 -21.99 
-14.00 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -14.00 
-21.99 -19.99 -14.00   0.00 

Iterate No.146
  0.00 -14.00 -19.99 -21.99 
-14.00 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -14.00 
-21.99 -19.99 -14.00   0.00 

Iterate No.147
  0.00 -14.00 -19.99 -21.99 
-14.00 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -14.00 
-21.99 -19.99 -14.00   0.00 

Iterate No.148
  0.00 -14.00 -19.99 -21.99 
-14.00 -17.99 -19.99 -19.99 
-19.99 -19.99 -17.99 -14.00 
-21.99 -19.99 -14.00   0.00 

Iterate No.149
  0.00 -14.00 -19.99 -21.99 
-14.00 -18.00 -19.99 -19.99 
-19.99 -19.99 -18.00 -14.00 
-21.99 -19.99 -14.00   0.00 

Iterate No.150
  0.00 -14.00 -19.99 -21.99 
-14.00 -18.00 -19.99 -19.99 
-19.99 -19.99 -18.00 -14.00 
-21.99 -19.99 -14.00   0.00 

Iterate No.151
  0.00 -14.00 -20.00 -21.99 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-21.99 -20.00 -14.00   0.00 

Iterate No.152
  0.00 -14.00 -20.00 -21.99 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-21.99 -20.00 -14.00   0.00 

Iterate No.153
  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00 

Iterate No.154
  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00 

Iterate No.155
  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00 

Iterate No.156
  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00 

Iterate No.157
  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00 

Iterate No.158
  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00 

Iterate No.159
  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00 

Iterate No.160
  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00 

  0.00 -14.00 -20.00 -22.00 
-14.00 -18.00 -20.00 -20.00 
-20.00 -20.00 -18.00 -14.00 
-22.00 -20.00 -14.00   0.00

相關推薦

RL實踐1——值迭代求解隨機策略

RL實踐1——值迭代求解隨機策略 參考自葉強《強化學習》第三講，方格世界—— 使用 動態規劃 求解隨機策略

RL實踐1——動態規劃值迭代

RL實踐1——值迭代求解隨機策略 參考自葉強《強化學習》第三講，方格世界—— 使用 動態規劃 求解隨機策略

binary-tree-postorder-traversa 迭代求解二叉樹後序遍歷

題目： 求給定的二叉樹的後序遍歷。 例如： 給定的二叉樹為{1,#,2,3}, 返回[3,2,1].

1、迭代器 Iterator模式 一個一個遍歷 行為型設計模式

1、Iterator模式 迭代器（iterator）有時又稱遊標（cursor）是程序設計的軟件設計模式，可在容器（container，例如鏈表或者陣列）上遍訪的接口，設計人員無需關心容器的內容。

Python學習筆記之4.10 - 序列上索引值迭代 》》》enumerate() 函式 》》》待續

問題： 你想在迭代一個序列的同時跟蹤正在被處理的元素索引。 解決方案: 內建的 enumerate() 函式可以很好的解決這個問題：

遞迴1 （迭代） 母牛問題

#include<bits/stdc++.h> using namespace std; int fun(int n) { if(n <= 4) return n; else { return fun(n - 1)+fun(n - 3);

08 Python 中的迭代器趣味實踐

上節課我們學習了迭代器的實現原理，這節課我們來動手實踐一下： 1. 遍歷文字檔案中的單詞

python解壓可迭代物件賦值給多個變數

如果一個可迭代物件的元素個數超過了變數的個數，會出現“解壓值太多”的異常，python提供*號表示式（碼夫破石的理解）來解決。

binary-tree-preorder-traversal 迭代法求解二叉樹前序遍歷

題目： 求給定的二叉樹的前序遍歷。 例如： 給定的二叉樹為{1,#,2,3}, 1　　2　　/　　3

關於雙線性插值中重疊畫素與空白畫素掩膜函式的一種迭代batch的寫法

1 from __future__ import division 2 import matplotlib.pyplot as plt 3 import numpy as np 4 import tensorflow as tf

基於Python的影象閾值化分割(迭代法)

1.閾值化分割原理 通過對影象的灰度直方圖進行數學統計，選擇一個或多個閾值將畫素劃分為若干類。一般情況下，當影象由灰度值相差較大的目標和背景組成時，如果目標區域內部畫素灰度分佈均勻一致，背景區域畫素在另一

c語言 牛頓迭代法求解高階一元方程

現在有一個任意的高階一元方程，x的冪可以任意數，求解該方程的解 例如，現在有x的三次方+2x+3，求解其根

07.1 迭代器、生成器

# 迭代器 \"\"\" 1.迭代器(Iterator)是訪問集合內元素的一種方式，一般用來遍歷資料

五點差分法求解泊松方程-共軛梯度法和G-S迭代法

技術標籤：傳統數值方法矩陣線性代數 差分法求解矩形區域橢圓方程 Δ u = f , x ∈ Ω

C++中的迭代器進階1：插入迭代器

技術標籤：C++語法 插入迭代器：insert iterator 標頭檔案iterator可以用來向容器插入元素。一種迭代器介面卡：關於介面卡支援的操作見下表：

LeetCode: 138. 複製帶隨機指標的連結串列(中等)[DFS, 迭代]

題目描述 給你一個長度為 n 的連結串列，每個節點包含一個額外增加的隨機指標 random ，該指標可以指向連結串列中的任何節點或空節點。

newton_method 牛頓迭代法求解

newton_method牛頓迭代法求解 參考： https://baike.baidu.com/item/牛頓迭代法/10887580?fromtitle=牛頓法&fromid=1384129&fr=aladdin

【每日一題】【連結串列&頭插法&ASCII碼】【連結串列&迭代器】2022年1月28日-NC1 大數加法

描述以字串的形式讀入兩個數字，編寫一個函式計算它們的和，以字串形式返回。

4.1手動遍歷迭代器

問題： ​你想遍歷一個可迭代物件中的所有元素，但是卻不想使用for迴圈 解決方案：

牛頓迭代法求解平方根

假設現在輸入一個整數，希望通過某種方式來求得該整數的平方根，要求得到儘可能大的精度。