1. 程式人生 > >torch.nn.CrossEntropyLoss的相關

torch.nn.CrossEntropyLoss的相關

 引數可以看下面

class CrossEntropyLoss(_WeightedLoss):

  def __init__(self, weight=None, size_average=True, ignore_index=-100, reduce=True):
    pass
  def forward(self, input, target):
    pass

解釋:網上的一些解釋,該損失的計算公式:https://blog.csdn.net/tmk_01/article/details/80839810

這裡面的公式的2個例子舉的很好,該例子是判斷每一個例項的分類,比如圖片是屬於哪一個類的。所以每一個例項的真實標籤是一維的。做不一樣的任務,例項的標籤也是不一樣的,有1維或者是多維的。舉例:如果輸入的是影象,任務是分類每個影象對應一個類別即數字是一維的,總共有class個類別,那麼input=【n,c,h,w】,target=【n】是一維的,元素個數為n;如果是做語義分割任務是得到一張【h,w】的語義圖,那麼input=【n,c,h,w】,target=【n,h,w】。

r"""This criterion combines :func:`nn.LogSoftmax` and :func:`nn.NLLLoss` in one single class.

    It is useful when training a classification problem with `C` classes.
    If provided, the optional argument :attr:`weight` should be a 1D `Tensor`
    assigning weight to each of the classes.
    This is particularly useful when you have an unbalanced training set.

    The `input` is expected to contain scores for each class.

    `input` has to be a Tensor of size either :math:`(minibatch, C)` or
    :math:`(minibatch, C, d_1, d_2, ..., d_K)`
    with :math:`K \geq 2` for the `K`-dimensional case (described later).

    This criterion expects a class index (0 to `C-1`) as the
    `target` for each value of a 1D tensor of size `minibatch`

    The loss can be described as:

    .. math::
        loss(x, class) = -x[class] + log(sum_j (exp(x[j])))

    or in the case of the `weight` argument being specified:

    .. math::
        loss(x, class) = weight[class] (-x[class] + log(sum_j (exp(x[j]))))

    The losses are averaged across observations for each minibatch.

    Can also be used for higher dimension inputs, such as 2D images, by providing
    an input of size :math:`(minibatch, C, d_1, d_2, ..., d_K)` with :math:`K \geq 2`,
    where :math:`K` is the number of dimensions, and a target of appropriate shape
    (see below).
"""

引數:

"""

    Args:
        weight (Tensor, optional): a manual rescaling weight given to each class.
           If given, has to be a Tensor of size `C`
        size_average (bool, optional): By default, the losses are averaged over observations for each minibatch.
           However, if the field `size_average` is set to ``False``, the losses are
           instead summed for each minibatch. Ignored if reduce is ``False``.
        ignore_index (int, optional): Specifies a target value that is ignored
            and does not contribute to the input gradient. When `size_average` is
            ``True``, the loss is averaged over non-ignored targets.
        reduce (bool, optional): By default, the losses are averaged or summed over
            observations for each minibatch depending on `size_average`. When reduce
            is ``False``, returns a loss per batch instead and ignores
            size_average. Default: ``True``

    Shape:
        - Input: :math:`(N, C)` where `C = number of classes`, or
            :math:`(N, C, d_1, d_2, ..., d_K)` with :math:`K \geq 2`
            in the case of `K`-dimensional loss.
        - Target: :math:`(N)` where each value is :math:`0 \leq \text{targets}[i] \leq C-1`, or
            :math:`(N, d_1, d_2, ..., d_K)` with :math:`K \geq 2` in the case of
            K-dimensional loss.
        - Output: scalar. If reduce is ``False``, then the same size
            as the target: :math:`(N)`, or
            :math:`(N, d_1, d_2, ..., d_K)` with :math:`K \geq 2` in the case
            of K-dimensional loss.

    Examples::

        >>> loss = nn.CrossEntropyLoss()
        >>> input = torch.randn(3, 5, requires_grad=True)
        >>> target = torch.empty(3, dtype=torch.long).random_(5)
        >>> output = loss(input, target)
        >>> output.backward()
"""

注意上面的target的型別一定是torch.long型別.還有情況要注意:如下圖

import torch.nn as nn
import torch.nn.functional as F
#可以的格式
#首先要定義這個函式,也就是例項化才能使用
loss = nn.CrossEntropyLoss()(input,target)

#這個函式已經進行了定義為criterion
criterion = nn.CrossEntropyLoss()
loss = criterion(input,target)

#F函式裡面有這個函式的相關定義,直接呼叫就可以
loss = F.cross_entropy(input,target)

#上述三個的輸出結果是一樣的,loss 型別為torch.Tensor是一個可求導的tensor; loss.item型別為float是python型別的常數值.

#不可以的格式
loss = nn.CrossEntropyLoss(input,target)
#得到的loss型別為torch.nn.modules.loss.CrossEntropyLoss,相當於幹函式的例項化,而不是上面的tensor型別

 也就是說loss是一個可求導的tensor常數,而loss.item是一個float型別的常數,不能進行求導

講解完了這個函式,接下來看一下具體的應用https://mp.csdn.net/postedit/82885569