1. 程式人生 > >detectron程式碼理解(七):FPN調參出現問題

detectron程式碼理解(七):FPN調參出現問題

首先根據:

def _narrow_to_fpn_roi_levels(blobs, spatial_scales):
    """Return only the blobs and spatial scales that will be used for RoI heads.
    Inputs `blobs` and `spatial_scales` may include extra blobs and scales that
    are used for RPN proposals, but not for RoI heads.
    """
    # Code only supports case when RPN and ROI min levels are the same
    assert cfg.FPN.RPN_MIN_LEVEL == cfg.FPN.ROI_MIN_LEVEL
    # RPN max level can be >= to ROI max level
    assert cfg.FPN.RPN_MAX_LEVEL >= cfg.FPN.ROI_MAX_LEVEL
    # FPN RPN max level might be > FPN ROI max level in which case we
    # need to discard some leading conv blobs (blobs are ordered from
    # max/coarsest level to min/finest level)
    num_roi_levels = cfg.FPN.ROI_MAX_LEVEL - cfg.FPN.ROI_MIN_LEVEL + 1
    return blobs[-num_roi_levels:], spatial_scales[-num_roi_levels:]

可以看出在使用FPN的條件下,RPN_MIN_LEVEL 必須等於 ROI_MIN_LEVEL,同時RPN_MAX_LEVEL必須大於等於ROI_MAX_LEVEL,不存在RPN_MAX_LEVEL = 4 < ROI_MAX_LEVEL = 5的情況

情況1(最基本的):

# Use FPN for RoI transform for object detection if True
__C.FPN.MULTILEVEL_ROIS = True
# Coarsest level of the FPN pyramid
__C.FPN.ROI_MAX_LEVEL = 5
# Finest level of the FPN pyramid
__C.FPN.ROI_MIN_LEVEL = 2

# Use FPN for RPN if True
__C.FPN.MULTILEVEL_RPN = True
# Coarsest level of the FPN pyramid
__C.FPN.RPN_MAX_LEVEL = 6
# Finest level of the FPN pyramid
__C.FPN.RPN_MIN_LEVEL = 2
# FPN RPN anchor aspect ratios

這種是最基本的情況,也就是對RPN和rois同時採用多尺度

 此時對resnet網路增加FPN層時,需要確定fpn的min level和max level,確定函式為:

def get_min_max_levels():
    """The min and max FPN levels required for supporting RPN and/or RoI
    transform operations on multiple FPN levels.
    """
    min_level = LOWEST_BACKBONE_LVL  #2
    max_level = HIGHEST_BACKBONE_LVL  #5
    if cfg.FPN.MULTILEVEL_RPN and not cfg.FPN.MULTILEVEL_ROIS:  #如果使用MULTILEVEL_RPN,但是不使用MULTILEVEL_ROIS
        max_level = cfg.FPN.RPN_MAX_LEVEL
        min_level = cfg.FPN.RPN_MIN_LEVEL
    if not cfg.FPN.MULTILEVEL_RPN and cfg.FPN.MULTILEVEL_ROIS:  #如果使用MULTILEVEL_ROIS,但是不使用MULTILEVEL_RPN
        max_level = cfg.FPN.ROI_MAX_LEVEL
        min_level = cfg.FPN.ROI_MIN_LEVEL
    if cfg.FPN.MULTILEVEL_RPN and cfg.FPN.MULTILEVEL_ROIS:      #如果兩者都使用
        max_level = max(cfg.FPN.RPN_MAX_LEVEL, cfg.FPN.ROI_MAX_LEVEL) #取兩者之中的最大,和兩者之中的最小
        min_level = min(cfg.FPN.RPN_MIN_LEVEL, cfg.FPN.ROI_MIN_LEVEL)
    return min_level, max_level

在兩者都為True的情況下,min_level 為兩者中的最小,即2,max_level為兩者中的最大,即6。所以此是增加P2-P6。

之後還要為每個fpn層增加RPN,需要呼叫的函式如下:

def add_fpn_rpn_outputs(model, blobs_in, dim_in, spatial_scales):
    """Add RPN on FPN specific outputs."""
      
    num_anchors = len(cfg.FPN.RPN_ASPECT_RATIOS) #針對FPN設定的RPN,每一層的相應位置只產生RPN_ASPECT_RATIOS個anchor,面積是固定的。隨著層數的增加,面積增加
    dim_out = dim_in  #經過FPN後,特徵的維度是256

    k_max = cfg.FPN.RPN_MAX_LEVEL  # coarsest level of pyramid 這裡為6
    k_min = cfg.FPN.RPN_MIN_LEVEL  # finest level of pyramid   這裡為2
    assert len(blobs_in) == k_max - k_min + 1 

 因為是為每一層fpn增加RPN,所以要去fpn的層數,要等於設定的增加RPN層數,即assert len(blobs_in) == k_max - k_min + 1

剛剛我們增加了P2-P6層的fpn,也就是len(blobs_in)  = 5,而k_max - k_min + 1 = 6 - 2+ 1=5,兩者相等所以可以執行。

情況2

把MULTILEVEL_RPN關掉,如下

# Use FPN for RoI transform for object detection if True
__C.FPN.MULTILEVEL_ROIS = True
# Coarsest level of the FPN pyramid
__C.FPN.ROI_MAX_LEVEL = 5
# Finest level of the FPN pyramid
__C.FPN.ROI_MIN_LEVEL = 2

# Use FPN for RPN if True
__C.FPN.MULTILEVEL_RPN = False
# Coarsest level of the FPN pyramid
__C.FPN.RPN_MAX_LEVEL = 6
# Finest level of the FPN pyramid
__C.FPN.RPN_MIN_LEVEL = 2
# FPN RPN anchor aspect ratios

儘管關掉了MULTILEVEL_RPN,但是RPN_MAX_LEVEL和RPN_MIN_LEVEL的數值仍然存在,所以在為fpn增加rpn的時候,len(blobs_in)  = 5,而k_max - k_min + 1 = 6 - 2+ 1=4,不會報錯。

但是僅僅是這裡不會報錯,如果你執行的話,報錯的位置會在FPN.py:

此時再把MULTILEVEL_RPN改為True,則就可以運行了,所以就感到非常費解,既然MULTILEVEL_RPN只能設定為True,那麼設定的意義是什麼?

情況3

把MULTILEVEL_ROIS關掉,如下

# Use FPN for RoI transform for object detection if True
__C.FPN.MULTILEVEL_ROIS = False
# Coarsest level of the FPN pyramid
__C.FPN.ROI_MAX_LEVEL = 5
# Finest level of the FPN pyramid
__C.FPN.ROI_MIN_LEVEL = 2

# Use FPN for RPN if True
__C.FPN.MULTILEVEL_RPN = True
# Coarsest level of the FPN pyramid
__C.FPN.RPN_MAX_LEVEL = 6
# Finest level of the FPN pyramid
__C.FPN.RPN_MIN_LEVEL = 2
# FPN RPN anchor aspect ratios

此時報錯的位置在:

原因也不是很清楚

情況4

在yaml檔案中選擇只選用最精細層

MODEL:
  TYPE: generalized_rcnn
  CONV_BODY: FPN.add_fpn_ResNet50_conv5_P2only_body
FPN:
  FPN_ON: False
  MULTILEVEL_ROIS: False
  MULTILEVEL_RPN: False

此時報錯

反正瞎搗鼓了半天,覺得:

  • MULTILEVEL_ROIS,MULTILEVEL_RPN,FPN_ON就保持True吧,別亂改了
  • RPN的層數要大於等於ROIS的層數