detectron程式碼理解(七):FPN調參出現問題
首先根據:
def _narrow_to_fpn_roi_levels(blobs, spatial_scales): """Return only the blobs and spatial scales that will be used for RoI heads. Inputs `blobs` and `spatial_scales` may include extra blobs and scales that are used for RPN proposals, but not for RoI heads. """ # Code only supports case when RPN and ROI min levels are the same assert cfg.FPN.RPN_MIN_LEVEL == cfg.FPN.ROI_MIN_LEVEL # RPN max level can be >= to ROI max level assert cfg.FPN.RPN_MAX_LEVEL >= cfg.FPN.ROI_MAX_LEVEL # FPN RPN max level might be > FPN ROI max level in which case we # need to discard some leading conv blobs (blobs are ordered from # max/coarsest level to min/finest level) num_roi_levels = cfg.FPN.ROI_MAX_LEVEL - cfg.FPN.ROI_MIN_LEVEL + 1 return blobs[-num_roi_levels:], spatial_scales[-num_roi_levels:]
可以看出在使用FPN的條件下,RPN_MIN_LEVEL 必須等於 ROI_MIN_LEVEL,同時RPN_MAX_LEVEL必須大於等於ROI_MAX_LEVEL,不存在RPN_MAX_LEVEL = 4 < ROI_MAX_LEVEL = 5的情況
情況1(最基本的):
# Use FPN for RoI transform for object detection if True __C.FPN.MULTILEVEL_ROIS = True # Coarsest level of the FPN pyramid __C.FPN.ROI_MAX_LEVEL = 5 # Finest level of the FPN pyramid __C.FPN.ROI_MIN_LEVEL = 2 # Use FPN for RPN if True __C.FPN.MULTILEVEL_RPN = True # Coarsest level of the FPN pyramid __C.FPN.RPN_MAX_LEVEL = 6 # Finest level of the FPN pyramid __C.FPN.RPN_MIN_LEVEL = 2 # FPN RPN anchor aspect ratios
這種是最基本的情況,也就是對RPN和rois同時採用多尺度。
此時對resnet網路增加FPN層時,需要確定fpn的min level和max level,確定函式為:
def get_min_max_levels(): """The min and max FPN levels required for supporting RPN and/or RoI transform operations on multiple FPN levels. """ min_level = LOWEST_BACKBONE_LVL #2 max_level = HIGHEST_BACKBONE_LVL #5 if cfg.FPN.MULTILEVEL_RPN and not cfg.FPN.MULTILEVEL_ROIS: #如果使用MULTILEVEL_RPN,但是不使用MULTILEVEL_ROIS max_level = cfg.FPN.RPN_MAX_LEVEL min_level = cfg.FPN.RPN_MIN_LEVEL if not cfg.FPN.MULTILEVEL_RPN and cfg.FPN.MULTILEVEL_ROIS: #如果使用MULTILEVEL_ROIS,但是不使用MULTILEVEL_RPN max_level = cfg.FPN.ROI_MAX_LEVEL min_level = cfg.FPN.ROI_MIN_LEVEL if cfg.FPN.MULTILEVEL_RPN and cfg.FPN.MULTILEVEL_ROIS: #如果兩者都使用 max_level = max(cfg.FPN.RPN_MAX_LEVEL, cfg.FPN.ROI_MAX_LEVEL) #取兩者之中的最大,和兩者之中的最小 min_level = min(cfg.FPN.RPN_MIN_LEVEL, cfg.FPN.ROI_MIN_LEVEL) return min_level, max_level
在兩者都為True的情況下,min_level 為兩者中的最小,即2,max_level為兩者中的最大,即6。所以此是增加P2-P6。
之後還要為每個fpn層增加RPN,需要呼叫的函式如下:
def add_fpn_rpn_outputs(model, blobs_in, dim_in, spatial_scales):
"""Add RPN on FPN specific outputs."""
num_anchors = len(cfg.FPN.RPN_ASPECT_RATIOS) #針對FPN設定的RPN,每一層的相應位置只產生RPN_ASPECT_RATIOS個anchor,面積是固定的。隨著層數的增加,面積增加
dim_out = dim_in #經過FPN後,特徵的維度是256
k_max = cfg.FPN.RPN_MAX_LEVEL # coarsest level of pyramid 這裡為6
k_min = cfg.FPN.RPN_MIN_LEVEL # finest level of pyramid 這裡為2
assert len(blobs_in) == k_max - k_min + 1
因為是為每一層fpn增加RPN,所以要去fpn的層數,要等於設定的增加RPN層數,即assert len(blobs_in) == k_max - k_min + 1
剛剛我們增加了P2-P6層的fpn,也就是len(blobs_in) = 5,而k_max - k_min + 1 = 6 - 2+ 1=5,兩者相等所以可以執行。
情況2
把MULTILEVEL_RPN關掉,如下
# Use FPN for RoI transform for object detection if True
__C.FPN.MULTILEVEL_ROIS = True
# Coarsest level of the FPN pyramid
__C.FPN.ROI_MAX_LEVEL = 5
# Finest level of the FPN pyramid
__C.FPN.ROI_MIN_LEVEL = 2
# Use FPN for RPN if True
__C.FPN.MULTILEVEL_RPN = False
# Coarsest level of the FPN pyramid
__C.FPN.RPN_MAX_LEVEL = 6
# Finest level of the FPN pyramid
__C.FPN.RPN_MIN_LEVEL = 2
# FPN RPN anchor aspect ratios
儘管關掉了MULTILEVEL_RPN,但是RPN_MAX_LEVEL和RPN_MIN_LEVEL的數值仍然存在,所以在為fpn增加rpn的時候,len(blobs_in) = 5,而k_max - k_min + 1 = 6 - 2+ 1=4,不會報錯。
但是僅僅是這裡不會報錯,如果你執行的話,報錯的位置會在FPN.py:
此時再把MULTILEVEL_RPN改為True,則就可以運行了,所以就感到非常費解,既然MULTILEVEL_RPN只能設定為True,那麼設定的意義是什麼?
情況3
把MULTILEVEL_ROIS關掉,如下
# Use FPN for RoI transform for object detection if True
__C.FPN.MULTILEVEL_ROIS = False
# Coarsest level of the FPN pyramid
__C.FPN.ROI_MAX_LEVEL = 5
# Finest level of the FPN pyramid
__C.FPN.ROI_MIN_LEVEL = 2
# Use FPN for RPN if True
__C.FPN.MULTILEVEL_RPN = True
# Coarsest level of the FPN pyramid
__C.FPN.RPN_MAX_LEVEL = 6
# Finest level of the FPN pyramid
__C.FPN.RPN_MIN_LEVEL = 2
# FPN RPN anchor aspect ratios
此時報錯的位置在:
原因也不是很清楚
情況4
在yaml檔案中選擇只選用最精細層
MODEL:
TYPE: generalized_rcnn
CONV_BODY: FPN.add_fpn_ResNet50_conv5_P2only_body
FPN:
FPN_ON: False
MULTILEVEL_ROIS: False
MULTILEVEL_RPN: False
此時報錯
反正瞎搗鼓了半天,覺得:
- MULTILEVEL_ROIS,MULTILEVEL_RPN,FPN_ON就保持True吧,別亂改了
- RPN的層數要大於等於ROIS的層數