概要

Torch 中提供了一種整理資料結構的好東西，叫做 DataLoader，可以用來包裝自己的資料，進行批訓練，而且批訓練可以有多種途徑

import torch
import torch.utils.data as Data

torch.manual_seed(1)     # reproducible

<torch._C.Generator at 0x106f673d8>

DataLoader

DataLoader 是 Torch 用來包裝資料的工具，如將 numpy array 等資料形式轉成 Tensor，然後放進該包裝器中。使用 DataLoader 可以有效迭代資料。
Data loader. Combines a dataset and a sampler, and provides single- or multi-process iterators over the dataset.

MINIBATCH_SIZE = 5    # mini batch size
x = torch.linspace(1, 10, 10)  # torch tensor
y = torch.linspace(10, 1, 10)

# first transform the data to dataset can be processed by torch
torch_dataset = Data.TensorDataset(data_tensor=x, target_tensor=y)
# put the dataset into DataLoader
loader = Data.DataLoader(
    dataset=torch_dataset,
    batch_size=MINIBATCH_SIZE,
    shuffle=True
 
,
    num_workers=2           # set multi-work num read data
)

for epoch in range(3):
    # 1 epoch go the whole data
    for step, (batch_x, batch_y) in enumerate(loader):
        # here to train your model
        print('\n\n epoch: ', epoch, '| step: ', step, '| batch x: ', batch_x.numpy(), '| batch_y: '
 
, batch_y.numpy())

 epoch:  0 | step:  0 | batch x:  [  4.   2.  10.   7.   3.] | batch_y:  [ 7.  9.  1.  4.  8.]


 epoch:  0 | step:  1 | batch x:  [ 1.  8.  6.  9.  5.] | batch_y:  [ 10.   3.   5.   2.   6.]


 epoch:  1 | step:  0 | batch x:  [ 10.   2.   7.   3.   4.] | batch_y:  [ 1.  9.  4.  8.  7.]


 epoch:  1 | step:  1 | batch x:  [ 5.  6.  9.  1.  8.] | batch_y:  [  6.   5.   2.  10.   3.]


 epoch:  2 | step:  0 | batch x:  [  7.   1.   4.   3.  10.] | batch_y:  [  4.  10.   7.   8.   1.]


 epoch:  2 | step:  1 | batch x:  [ 5.  8.  2.  6.  9.] | batch_y:  [ 6.  3.  9.  5.  2.]

可以看出，每個 step 取出 MINIBATCH_SIZE(這裡為5)個數據進行處理，而且每次 epoch 都是不一樣的，可知資料進行了 shuffle

資料經過 mini-batch 後，比每次迭代只訓練一個樣本要快，因為這樣可以利用 CPU 或 GPU 進行平行計算來 speed up，同時也比每次迭代全體樣本要好，因為如果一次迭代全體樣本來訓練的話，那麼每次迭代只能進行一次 forward propagation 和 backword propagation，會比較耗時，特別是對於資料量比較大時，會更糟糕

如果資料不能被 MINIBATCH_SIZE 整除會怎樣呢？最後的 step 返回剩餘的就好了，下面演示說明

MINIBATCH_SIZE = 8
# put the dataset into DataLoader
loader = Data.DataLoader(
    dataset=torch_dataset,
    batch_size=MINIBATCH_SIZE,
    shuffle=True,
    num_workers=2           # set multi-work num read data
)

for epoch in range(3):
    for step, (batch_x, batch_y) in enumerate(loader):
        print('\n\n epoch: ', epoch, '| step: ', step, '| batch x: ', batch_x.numpy(), '| batch y: ', batch_y.numpy())

 epoch:  0 | step:  0 | batch x:  [ 9.  1.  3.  8.  6.  7.  4.  2.] | batch y:  [  2.  10.   8.   3.   5.   4.   7.   9.]


 epoch:  0 | step:  1 | batch x:  [ 10.   5.] | batch y:  [ 1.  6.]


 epoch:  1 | step:  0 | batch x:  [  7.   1.   8.   3.   9.   2.   5.  10.] | batch y:  [  4.  10.   3.   8.   2.   9.   6.   1.]


 epoch:  1 | step:  1 | batch x:  [ 4.  6.] | batch y:  [ 7.  5.]


 epoch:  2 | step:  0 | batch x:  [  3.   1.   9.   6.   5.   7.  10.   2.] | batch y:  [  8.  10.   2.   5.   6.   4.   1.   9.]


 epoch:  2 | step:  1 | batch x:  [ 4.  8.] | batch y:  [ 7.  3.]