torch.nn.CrossEntropyLoss的相關
阿新 • • 發佈:2018-11-10
引數可以看下面
class CrossEntropyLoss(_WeightedLoss):
def __init__(self, weight=None, size_average=True, ignore_index=-100, reduce=True):
pass
def forward(self, input, target):
pass
解釋:網上的一些解釋,該損失的計算公式:https://blog.csdn.net/tmk_01/article/details/80839810
這裡面的公式的2個例子舉的很好,該例子是判斷每一個例項的分類,比如圖片是屬於哪一個類的。所以每一個例項的真實標籤是一維的。做不一樣的任務,例項的標籤也是不一樣的,有1維或者是多維的。舉例:如果輸入的是影象,任務是分類每個影象對應一個類別即數字是一維的,總共有class個類別,那麼input=【n,c,h,w】,target=【n】是一維的,元素個數為n;如果是做語義分割任務是得到一張【h,w】的語義圖,那麼input=【n,c,h,w】,target=【n,h,w】。
r"""This criterion combines :func:`nn.LogSoftmax` and :func:`nn.NLLLoss` in one single class. It is useful when training a classification problem with `C` classes. If provided, the optional argument :attr:`weight` should be a 1D `Tensor` assigning weight to each of the classes. This is particularly useful when you have an unbalanced training set. The `input` is expected to contain scores for each class. `input` has to be a Tensor of size either :math:`(minibatch, C)` or :math:`(minibatch, C, d_1, d_2, ..., d_K)` with :math:`K \geq 2` for the `K`-dimensional case (described later). This criterion expects a class index (0 to `C-1`) as the `target` for each value of a 1D tensor of size `minibatch` The loss can be described as: .. math:: loss(x, class) = -x[class] + log(sum_j (exp(x[j]))) or in the case of the `weight` argument being specified: .. math:: loss(x, class) = weight[class] (-x[class] + log(sum_j (exp(x[j])))) The losses are averaged across observations for each minibatch. Can also be used for higher dimension inputs, such as 2D images, by providing an input of size :math:`(minibatch, C, d_1, d_2, ..., d_K)` with :math:`K \geq 2`, where :math:`K` is the number of dimensions, and a target of appropriate shape (see below). """
引數:
""" Args: weight (Tensor, optional): a manual rescaling weight given to each class. If given, has to be a Tensor of size `C` size_average (bool, optional): By default, the losses are averaged over observations for each minibatch. However, if the field `size_average` is set to ``False``, the losses are instead summed for each minibatch. Ignored if reduce is ``False``. ignore_index (int, optional): Specifies a target value that is ignored and does not contribute to the input gradient. When `size_average` is ``True``, the loss is averaged over non-ignored targets. reduce (bool, optional): By default, the losses are averaged or summed over observations for each minibatch depending on `size_average`. When reduce is ``False``, returns a loss per batch instead and ignores size_average. Default: ``True`` Shape: - Input: :math:`(N, C)` where `C = number of classes`, or :math:`(N, C, d_1, d_2, ..., d_K)` with :math:`K \geq 2` in the case of `K`-dimensional loss. - Target: :math:`(N)` where each value is :math:`0 \leq \text{targets}[i] \leq C-1`, or :math:`(N, d_1, d_2, ..., d_K)` with :math:`K \geq 2` in the case of K-dimensional loss. - Output: scalar. If reduce is ``False``, then the same size as the target: :math:`(N)`, or :math:`(N, d_1, d_2, ..., d_K)` with :math:`K \geq 2` in the case of K-dimensional loss. Examples:: >>> loss = nn.CrossEntropyLoss() >>> input = torch.randn(3, 5, requires_grad=True) >>> target = torch.empty(3, dtype=torch.long).random_(5) >>> output = loss(input, target) >>> output.backward() """
注意上面的target的型別一定是torch.long型別.還有情況要注意:如下圖
import torch.nn as nn
import torch.nn.functional as F
#可以的格式
#首先要定義這個函式,也就是例項化才能使用
loss = nn.CrossEntropyLoss()(input,target)
#這個函式已經進行了定義為criterion
criterion = nn.CrossEntropyLoss()
loss = criterion(input,target)
#F函式裡面有這個函式的相關定義,直接呼叫就可以
loss = F.cross_entropy(input,target)
#上述三個的輸出結果是一樣的,loss 型別為torch.Tensor是一個可求導的tensor; loss.item型別為float是python型別的常數值.
#不可以的格式
loss = nn.CrossEntropyLoss(input,target)
#得到的loss型別為torch.nn.modules.loss.CrossEntropyLoss,相當於幹函式的例項化,而不是上面的tensor型別
也就是說loss是一個可求導的tensor常數,而loss.item是一個float型別的常數,不能進行求導
講解完了這個函式,接下來看一下具體的應用https://mp.csdn.net/postedit/82885569