【2017-2018】COCO Challenges分類整理
本文記錄了博主在研究COCO（Common Objects in Context） Challenges過程中關於挑戰賽分類的筆記。更新於2018.12.21。
官網中包括所有talk的ppt。
【2017-2018】COCO Challenges分類整理
2018挑戰賽
挑戰賽日期：
Workshop時間： 2018年9月9
COCO 2018挑戰賽包括：
相比較2017挑戰賽，2018挑戰賽在內容上做了改動。具體如下：
- 以segmentation masks的目標識別（例項分割）；
- 全景分割；
- 人體關鍵點估計；
- 稠密姿態。
The specific tracks in the COCO 2018 Challenges are (1) object detection with segmentation masks (instance segmentation), (2) panoptic segmentation, (3) person keypoint estimation, and (4) DensePose. We describe each next. Note: neither object detection with bounding-box outputs nor stuff segmentation will be featured at the COCO 2018 challenge (but evaluation servers for both tasks remain open).
Speaker：
下面是每個類別的詳細介紹。
COCO檢測挑戰
The COCO Object Detection Task is designed to push the state of the art in object detection forward. Note: only the detection task with object segmentation output (that is, instance segmentation) will be featured at the COCO 2018 challenge. For full details of this task please see the COCO Object Detection Task.
與2017年不同，2018年的挑戰賽僅接收以目標分割（例項分割）為輸出的演算法。
COCO針對兩類目標識別任務：
- bounding box output
- object segmentation output（instance segmentation）
但COCO2018僅接收分割類輸出的演算法。
演算法評價指標：
具體指標等參看detection evaluation頁面。
COCO的訓練、驗證和測試集包含超過200,000張圖片和80個目標類別，可以從下載介面下載。所有的例項都被詳細標註了segmentation mask，但只公開訓練集和驗證集（超500,000個已分割目標例項）的標註。2018年所使用的資料、度量和指導都與2017年目標識別任務的相同。二者唯一的區別是，2018年只接收分割類結果。
COCO測試集分為兩個部分：test-dev和test-challenge。其中在挑戰結束後，test-dev作為預設測試集用於維護公開的leaderboard，而test-challenge只用於挑戰賽，結果在workshop上公佈。
COCO2018檢測挑戰網址上還說明了提交資料的格式、上傳和評估要求。
COCO提供用於資料、標註和評估程式碼的API，可以從他們的GitHub上下載。API的使用說明在下載介面可以找到。COCO還提供了所有步驟的說明，包括下載、資料型別、結果格式、指導、上傳和評估。
COCO全景分割挑戰
COCO把檢測任務分為兩類：thing（人、車、象）和stuff（玻璃、牆、天空）。新出的全景任務（panoptic task）同時包含上面兩類。
The COCO Panoptic Segmentation Task has the goal of advancing the state of the art in scene segmentation. Panoptic segmentation addresses both stuff and thing classes, unifying the typically distinct semantic and instance segmentation tasks. For full details of this task please see the COCO Panoptic Segmentation Task.
全景挑戰資料庫和該資料庫對應的論文。資料庫中包括80個來自於檢測任務的thing類別和91個來自於stuff任務的stuff類別。關於全景分割的更多內容可以看這篇論文。
具體而言，things類別中是包括可數的目標，如人、動物、工具等；而stuff類別中是具有相似材質或紋理的無固定形態的一個區域，比如玻璃、天空、路等等。
The definition of ‘panoptic’ is “including everything visible in one view”, in our context panoptic refers to a unified, global view of segmentation.
致力於的應用場景：autonomous driving、augmented reality等。
評估標準：
具體評估內容可以參見全景分割評估網址。其中包括了類別說明、質量檢測公式、度量和評價程式碼等。
目前，COCO API不支援全景分割下的評估，但是可以從這裡找到資料、評估程式碼和評估服務，leaderboard也可以找到。
COCO關鍵點檢測
The COCO Keypoint Detection Task requires localization of person keypoints in challenging, uncontrolled conditions. The keypoint task involves simultaneously detecting people and localizing their keypoints (person locations are not given at test time). For full details of this task please see the COCO Keypoint Detection Task.
COCO關鍵點檢測要求演算法能夠在有挑戰、不受控的條件下檢測出人的關鍵點。關鍵點檢測任務包括人物檢測和其關鍵點定位（測試時不提供人的位置資訊）兩項工作。
評估標準：
具體見全景風格評估網址。
資料集包括超200,000張圖片和250,000個標註有關鍵點資訊的人物例項（COCO中大部分人物都是中等或大尺寸），這裡是下載地址。其中，訓練集和驗證集（超150,000個人和170萬已標註關鍵點）的標註是公開的。
COCO稠密姿態檢測
The COCO DensePose Task requires localization of dense person keypoints in challenging, uncontrolled conditions. The DensePose task involves simultaneously detecting people and localizing their dense keypoints, mapping all human pixels to a 3D surface of the human body. For full details of this task please see the COCO DensePose Task.
COCO2018稠密姿態檢測網址。其中包括資料集的下載地址和DensePose-RCNN系統說明等資訊。
Mapillary Challenges
2018年，Mapillary Research攜Mapillary Vistas資料庫加入了COCO識別任務。
Vistas is a diverse, pixel-accurate street-level image dataset for empowering autonomous mobility and transport at global scale. It has been designed and collected to cover diversity in appearance, richness of annotation detail, and geographic extent. The Mapillary challenges are based on the publicly available Vistas Research dataset, featuring:
- 28 stuff classes, 37 thing classes (w instance-specific annotations), and 1 void class
- 25K high-resolution images (18K train, 2K val, 5K test; w average resolution of ~9 megapixels)
- Global geographic coverage including North and South America, Europe, Africa, Asia, and Oceania
Highly variable weather conditions (sun, rain, snow, fog, haze) and capture times (dawn, daylight, dusk, night)- Broad range of camera sensors, varying focal length, image aspect ratios, and different types of camera noise
- Different capturing viewpoints (road, sidewalks, off-road)
基於Mapillary Vistas資料庫將會被分別歸入目標識別和全景分割這兩類任務中：
Challenge tracks based on the Mapillary Vistas dataset will be (1) object detection with segmentation masks (instance segmentation) and (2) panoptic segmentation, in line with COCO’s detection and panoptic segmentation tasks, respectively.
Mapillary Vistas 目標識別任務
The Mapillary Vistas Object Detection Task emphasizes recognizing individual instances of both static street-image objects (like street lights, signs, poles) but also dynamic street participants (like cars, pedestrians, cyclists). This task aims to push the state-of-the-art in instance segmentation, targeting critical perception tasks for autonomously acting agents like cars or transportation robots. For full details of this task please see the Mapillary Vistas Object Detection Task.
Mapillary Vistas全景分割任務
The Mapillary Vistas Panoptic Segmentation Task targets the full perception stack for scene segmentation in street-images. Panoptic segmentation addresses both stuff and thing classes, unifying the typically distinct semantic and instance segmentation tasks. For full details of this task please see the Mapillary Vistas Panoptic Segmentation Task.
2017挑戰賽
舉行時間：
Workshop時間： 2017年10月29
COCO 2017挑戰賽包括：
- 以bounding boxes和segmentation masks實現的目標檢測；
- 關節檢測和人類關鍵點估計；
- 實物分割。
The specific tracks in the COCO 2017 Challenges are (1) object detection with bounding boxes and segmentation masks, (2) joint detection and person keypoint estimation, and (3) stuff segmentation. We describe each next.
Speakers：
下面是每個類別的詳細介紹。
COCO檢測挑戰
The COCO 2017 Detection Challenge is designed to push the state of the art in object detection forward. Teams are encouraged to compete in either (or both) of two object detection challenges: using bounding box output or object segmentation output. For full details of this task please see the COCO Detection Challenge page.
COCO關鍵點挑戰
The COCO 2017 Keypoint Challenge requires localization of person keypoints in challenging, uncontrolled conditions. The keypoint challenge involves simultaneously detecting people and localizing their keypoints (person locations are not given at test time). For full details of this task please see the COCO Keypoints Challenge page.
注意，對於這一類挑戰，人的位置在測試時是不提供的，也就是說演算法自身要具備估計目標位置的能力。
COCO實物挑戰
The COCO 2017 Stuff Segmentation Challenge is designed to push the state of the art in semantic segmentation of stuff classes. Whereas the COCO 2017 Detection Challenge addresses thing classes (person, car, elephant), this challenge focuses on stuff classes (grass, wall, sky). For full details of this task please see the COCO Stuff Challenge page.
檢測挑戰主要針對的是thing（如人、車、大象等），而實物挑戰針對的是stuff（如杯子、牆、天空等）。
場景挑戰
The Places Challenge will host three tracks meant to complement the COCO Challenges. The data for the 2017 Places Challenge is from the pixel-wise annotated image dataset ADE20K, in which there are 20K images for training, 2K validation images, and 3K testing images. The three specific tracks in the Places Challenge 2017 are: (1) scene parsing, (2) instance segmentation, and (3) semantic boundary detection. See the Places Challenge Page for detailed information.
其他相關挑戰賽
VQA挑戰賽
