本文提出全卷積網路（FullyConvolutionalNetworks，FCN）用於影象語義分割。FCN主要思想是將一般的分類網路（如VGG，ResNet等）最後幾層的全連線層替換成卷積層。FCN的好處是可以接受任意尺寸的輸入影象。

1. image經過多個conv和+一個max pooling變為pool1 feature，寬高變為1/2
2. pool1 feature再經過多個conv+一個max pooling變為pool2 feature，寬高變為1/4
3. pool2 feature再經過多個conv+一個max pooling變為pool3 feature，寬高變為1/8
4. ......
5. 直到pool5 feature，寬高變為1/32。

• 對於FCN-32s，直接對pool5 feature進行32倍上取樣獲得32x upsampled feature，再對32x upsampled feature每個點做softmax prediction獲得32x upsampled feature prediction（即分割圖）。
• 對於FCN-16s，首先對pool5 feature進行2倍上取樣獲得2x upsampled feature，再把pool4 feature和2x upsampled feature逐點相加（element-wiseadd），然後對相加的feature進行16倍上取樣，並softmax prediction，獲得16x upsampled feature prediction。
• 對於FCN-8s，首先進行pool4+2x upsampled feature逐點相加，然後又進行pool3+2x upsampled逐點相加，即進行更多次特徵融合。具體過程與16s類似，不再贅述。

import
 
 torch
import torch.nn as nn
import torch.nn.init as init
import torch.nn.functional as F

from torch.utils import model_zoo
from torchvision import models

class FCN8(nn.Module):
    def __init__(self, num_classes):
        super().__init__()

        feats = list(models.vgg16(pretrained=True).features.children())

        self.feats = nn.Sequential(*feats[0:9])
        self.feat3 = nn.Sequential(*feats[10:16])
        self.feat4 = nn.Sequential(*feats[17:23])
        self.feat5 = nn.Sequential(*feats[24:30])

        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                m.requires_grad = False

        self.fconn = nn.Sequential(
            nn.Conv2d(512, 4096, 7),
            nn.ReLU(inplace=True),
            nn.Dropout(),
            nn.Conv2d(4096, 4096, 1),
            nn.ReLU(inplace=True),
            nn.Dropout(),
        )
        self.score_feat3 = nn.Conv2d(256, num_classes, 1)
        self.score_feat4 = nn.Conv2d(512, num_classes, 1)
        self.score_fconn = nn.Conv2d(4096, num_classes, 1)

    def forward(self, x):
        feats = self.feats(x)
        feat3 = self.feat3(feats)
        feat4 = self.feat4(feat3)
        feat5 = self.feat5(feat4)
        fconn = self.fconn(feat5)

        score_feat3 = self.score_feat3(feat3)
        score_feat4 = self.score_feat4(feat4)
        score_fconn = self.score_fconn(fconn)

        score = F.upsample_bilinear(score_fconn, score_feat4.size()[2:])
        score += score_feat4
        score = F.upsample_bilinear(score, score_feat3.size()[2:])
        score += score_feat3

        return F.upsample_bilinear(score, x.size()[2:])

• 最終的輸出通道數為21，為PASCAL資料集20類+1類背景。
• 網路最終的輸出大小為輸入影象width *輸入影象height * 21，損失函式是對每一個畫素點求softmaxloss，然後求和。
• 上取樣使用反捲積（deconvolution）的方式，使用雙線性插值初始化。
• 原網路中會設定第一層卷積層的pad=100，後面在特徵融合時引入了crop層。

其中， nij表示將本屬於第i類的畫素預測為屬於第j類的畫素數量；ncl表示畫素的類別總數；ti表示屬於第i類的畫素總數，