Detectron訓練日誌視覺化
阿新 • • 發佈：2018-12-11
Detectron框架的訓練日誌如下(一般還含有其它資訊，但是出於隱私方面的考慮刪去)
json_stats: {"accuracy_cls": 0.000000, "eta": "29 days, 16:00:50", "iter": 0, "loss": 6.594362, "loss_bbox": 0.024408, "loss_cls": 5.633613, "loss_rpn_bbox_fpn2": 0.094387, "loss_rpn_bbox_fpn3": 0.054846, "loss_rpn_bbox_fpn4": 0.023384, "loss_rpn_bbox_fpn5": 0.041790, "loss_rpn_bbox_fpn6" 以下指令碼使用matlibplot根據日誌進行視覺化，並儲存為圖片.指令碼在日誌檔案含有其他資訊的時候能夠正常工作.
import json
import re
from pylab import *
fig = figure(figsize=(8,6), dpi=300)
y1 = fig.add_subplot(111)
y1.set_xlabel('Iterations')
y2 = y1.twinx()
y1.set_ylim(0,1.0)
with open('./日誌名.txt') as f:
whole = f.read()
pattern = re.compile(r'json_stats: (\{.*\})')
lis = pattern.findall(whole)
try:
parsed = [json.loads(j) for j in lis]
print(parsed[0])
except:
print("json format is not corrrect")
exit(1)
_iter = [ j['iter'] for j in parsed]
_loss = [ j['loss'] for j in parsed]
_loss_bbox = [ j['loss_bbox'] for j in parsed]
_loss_cls = [ j['loss_cls'] for j in parsed]
_accuracy_cls = [ j['accuracy_cls'] for j in parsed]
_lr = [ j['lr'] for j in parsed]
try:
_mask_loss = [ j['mask_loss'] for j in parsed]
except:
_mask_loss = None
y1.plot(_iter, _loss_bbox, color="green", linewidth=1.0,linestyle="-",label='loss_bbox')
y1.plot(_iter, _loss, color="blue", linewidth=1.0, linestyle="-",label='loss')
y1.plot(_iter, _loss_cls, color="black", linewidth=1.0, linestyle="-",label='loss_cls')
y1.plot(_iter, _accuracy_cls, color="red", linewidth=1.0, linestyle="-",label='accuracy_cls')
if _mask_loss is not None:
y1.plot(_iter, _mask_loss, color="grey", linewidth=1.0, linestyle="-",label='mask_loss')
y2.set_ylim(0,max(_lr)/0.8)
y2.plot(_iter, _lr, color="purple", linewidth=1.0, linestyle="-",label='lr')
y2.set_ylabel('lr')
#可以選擇開啟網格
#grid()
#圖例
y1.legend()
savefig('./fig.png')
show()效果如下：
如何將訓練日誌儲存成檔案
Detectron在訓練網路的時候,會在終端中輸出訓練日誌,但是訓練日誌不會自動儲存成檔案.
只需要將輸出到終端的日誌,以管道的方式送至tee命令即可.
下面的命令將終端的輸出存成名為log.txt,檔名和字尾名其實不重要.
python2 tools/train_net.py \
--cfg configs/getting_started/tutorial_1gpu_e2e_faster_rcnn_R-50-FPN.yaml \
OUTPUT_DIR /tmp/detectron-output | tee log.txt
使用tee方式比起重定向至檔案的方式來說更加方便:因為它同時保留了終端中的輸出.