【Numberical Optimization】4 Trust-Region Methods (zen學習筆記)

阿新 • • 發佈：2018-12-13

TR方法比LS方法收斂速度快

TR方法中有幾個引數需要選擇：

近似模型 $m_{k}$
可信賴區域 $\Delta _{k}$
求解引數 $p_{k}$

$\mathbf{m_{k}}$

when $\alpha =1$ ，Taylor-series expansion of f around $x_{k}$ ，which is

$f(x_{k}+p)=f({x_{k}})+\bigtriangledown f(x_{k})p+\frac{1}{2}p^{T}\bigtriangledown ^{2}f(x+tp)p$

where t is some scalar in the interval (0,1).

By using an approximation $B_{k}$ to the Hessian in the second-order term:

$m_{k}(p)=f_{k}+g_{k}^{T}p+\frac{1}{2}p^{T}B_{k}p$

Then we seek a solution of subproblem:

${\color{Blue} min } m_{k}(p)=f_{k}+g_{k}^{T}p+\frac{1}{2}p^{T}B_{k}p$

$s.t. \left \| p \right \| <\bigtriangleup _{k}$

The difference between $m_{k}(p)$ and $f(x_{k}+p)$ is $O(\left \| p \right \|^{2})$ , which is small when p is small.

$\mathbf{\bigtriangleup _{k}}$

Let

$\rho _{k}=\frac{f(x_{k})-f(x_{k}+p_{k})}{m_{k}(0)-m_{k}(p_{k})}$

1.if $\rho _{k}$ is negative , the newe value $f_{k+1}$ is greater than $f_{k}$ , so the step must be rejected.,because step $p_{k}}$ is obtained by minimizing the model $m_{k}$ .

2.if $\rho _{k}$ is close to 1, so it safe to expand the trust region.

3.if $\rho _{k}$ is postive but significantly smaller than 1,we do not alter the trust region.

4.if $\rho _{k}$ is close to 0, we shrink the trust region.

專注於求解子問題：

We sometimes drop the interation subscript k and restate the problem as follows:

${\color{Cyan} min} m(p)=f+g^{T}p+\frac{1}{2}p^{T}Bp$

$s.t. \left \| p \right \|\leq \bigtriangleup$

if and only if

(4.8b) is a complementarity condition that states at least one of $\lambda$ and ( $\bigtriangleup -\left \| p^{*} \right \|$ ) must be 0.

When $\bigtriangleup =\bigtriangleup _{1}$ ， $p^{*3}$ lies strictly inside the trust region，we must have $\lambda =0$ .

When $\bigtriangleup =\bigtriangleup _{2}$ or $\bigtriangleup _{3}$ , we have $\bigtriangleup -\left \| p^{*} \right \|=0$ , then we get

$\lambda p^{*}=-Bp^{*}-g=-\bigtriangledown m(p^{*})$

Finally we get p.

【Numberical Optimization】4 Trust-Region Methods (zen學習筆記)

專注於求解子問題：

We sometimes drop the interation subscript k and restate the problem as follows:

4.1 Algotithms based on the Cauchy point

【Numberical Optimization】4 Trust-Region Methods (zen學習筆記)

【幹貨】JavaScript DOM編程藝術學習筆記4-6

【複雜網路】複雜網路分析庫NetworkX學習筆記（4）：統計指標計算

【計算機網絡】一個小白的DNS學習筆記 (＞﹏＜）

【雜題集】【51NOD 1267】4個數和為0

【ArcGIS操作】4 空間分析篇

【C++11】4種智慧指標

【資料倉庫】4. 拉鍊表

【設計經驗】4、SERDES關鍵技術總結

【R語言】4種R包安裝方式

【JavaScript高階】4、基礎總結深入（函式）

【資料庫篇】——4.使用者登入註冊系統設計和分析思路

【OpenCV數字影象處理】(4) 【Python程式設計】(4) 平移

【sklearn例項】4--特徵工程之離散值編碼

【MarketAnalysis總結】4.0使用者登入等賬戶安全訪問的實現

【重溫基礎】4.函式

【ECMAScript 5_6_7】4、ES6——模板字串

【資料結構】4.1串的基本操作（附程式碼實現）

【執行緒】- 4種執行緒通訊方式

【問答集錦】人工智能／機器學習技術在電商場景下的應用

【Numberical Optimization】4 Trust-Region Methods (zen學習筆記)

專注於求解子問題：

We sometimes drop the interation subscript k and restate the problem as follows:

4.1 Algotithms based on the Cauchy point

相關推薦