論文閱讀筆記ECCV 2018: Factorizable net: an efficient subgraph-based framework for scene graph generation
一、contribution
提出了一種基於子圖的場景圖生成方法,該方法具有以下特點:
(1)首先,提出了一種自底向上的聚類方法,將影象分解為子圖。通過共享子圖中的區域表示,我們的方法可以顯著減少冗餘計算並加快推理速度。此外,較少的表示允許我們使用二維特徵圖來維護子圖區域的空間資訊。
(2)其次,提出了一種空間加權訊息傳遞(SMP)結構,用於在物件特徵向量和子圖特徵對映之間傳遞訊息。
(3)第三,提出了一個空間敏感關係推理(SRI)模組,該模組利用主語、賓語和子圖表示的特徵來識別物件之間的關係。視覺關係檢測和視覺基因組的實驗表明,我們的方法優於最先進的方法,推理速度顯著加快。
二、method
步驟:
(1)generate object region proposals with RPN(region proposal network)
(2)group the object proposals into pairs and establish the fully-connected relations
(3)cluster the fully-connected graph into several subgraphs and share the subgroup features for object pairs within the subgraph, then a factorized connection graph(映像連線圖)is obtained by treating each subgraph as a node
(4)ROI pools the objects and subgraph features and transforms them into feature vectors and 2D feature maps respectively
(5) jointly refine the object and subgraph featuresby passing message along the subgraph-based connection graph for better rep-resentations
(6) recognize the object categories with object features and theirrelations (predicates) by fusing the subgraph features and object feature pairs