• nn.Conv2d的groups引數：
  groups引數控制分組卷積，引數預設為1，即普通二維卷積。
  當groups=1時：

  conv = nn.Conv2d(in_channels=6, out_channels=6, kernel_size=1, groups=1)
  conv.weight.data.size()
  #torch.Size([6, 6, 1, 1])
  

  當groups=2時：

  conv = nn.Conv2d(in_channels=6, out_channels=6, kernel_size=1, groups=2)
  conv.weight.data.size()
  #torch.Size([6, 3, 1, 1])
 

當groups=in_channels時：

  conv = nn.Conv2d(in_channels=6, out_channels=6, kernel_size=1, groups=6)
  conv.weight.data.size()
  #torch.Size([6, 1, 1, 1])

out_channels必須能被groups整除。

• 由於引數初始化不同，訓練結果差異很大。固定引數方法如下

  #cudnn確保精度。實際上影響不大，會導致計算效率降低
  from torch.backends import cudnn
  cudnn.benchmark = False            # if benchmark=True, deterministic will be False
  cudnn.deterministic = True
  
  #pytorch設定隨機種子
  torch.manual_seed(seed)            # 為CPU設定隨機種子
  torch.cuda.manual_seed(seed)       # 為當前GPU設定隨機種子
  torch.cuda.manual_seed_all(seed)   # 為所有GPU設定隨機種子
  
  #資料讀取中的隨即預處理。可對python,numpy設定隨機種子
  import random
  import numpy as np
  random.seed(seed)
  np.random.seed(seed)
  
  #dataloader中的讀取順序執行結果也會有差異
 

• 論文中最後一層是sotfmax+fc，然而在程式碼最後一層中加入sotfmax反而損失不下降。查閱官方文件發現是由於損失函式採用cross_entropy，而cross_entropy中這麼描述his criterion combines log_softmax and nll_loss in a single function.，實現程式碼如下：

  def cross_entropy(input, target, weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='mean'):
      if not torch.jit.is_scripting():
          tens_ops = (input, target)
          if any([type(t) is not Tensor for t in tens_ops]) and has_torch_function(tens_ops):
              return handle_torch_function(
                  cross_entropy, tens_ops, input, target, weight=weight,
                  size_average=size_average, ignore_index=ignore_index, reduce=reduce,
                  reduction=reduction)
      if size_average is not None or reduce is not None:
          reduction = _Reduction.legacy_get_string(size_average, reduce)
      return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
 

總結：如果損失函式採用nn.CrossEntropyLoss()，那麼最後一層不需要加softmax。