tfrecords

由於有個5W張圖片的訓練集，如果是placeholder的方式來把資料餵給graph的話，那可真蠢。。所以不如用tensorflow官方推薦的tfrecords來進行IO。這種IO方式主要是兩步：
1.先把資料dump成tfrecords檔案
2.用佇列來把資料餵給模型

寫資料

tfrecords檔案是一種二進位制檔案，雖然如果把這個二進位制檔案開啟會發現完全沒有可讀性，但是tensorflow在讀寫這種二進位制檔案的時候底層用的是google推行的protocol buffer（協議記憶體），這就比較舒服了，完全可以把二進位制檔案當成黑盒子，程式碼層面邏輯正確的話就沒問題。

剛剛說到tfrecords其實是protocol buffer，那不如捋一捋它的protocol。首先可以想象一下資料集是由很多條資料組成的，每條資料是由很多個欄位（特徵）組成的。所以tfrecords中用example來表示一條資料，用features來表示一條資料中的特徵。

那麼就可以順藤摸瓜了，來看看example.proto和feature.proto長啥樣。

example.proto

// Protocol messages for describing input data Examples for machine learning
// model training or inference.
syntax = "proto3";

import "tensorflow/core/example/feature.proto";
option cc_enable_arenas = true;
option java_outer_classname = "ExampleProtos";
option java_multiple_files = true;
option java_package = "org.tensorflow.example";

package tensorflow;

// An Example is a mostly-normalized data format for storing data for
// training and inference.  It contains a key-value store (features); where
// each key (string) maps to a Feature message (which is oneof packed BytesList,
// FloatList, or Int64List).  This flexible and compact format allows the
// storage of large amounts of typed data, but requires that the data shape
// and use be determined by the configuration files and parsers that are used to
// read and write this format.  That is, the Example is mostly *not* a
// self-describing format.  In TensorFlow, Examples are read in row-major
// format, so any configuration that describes data with rank-2 or above
// should keep this in mind.  For example, to store an M x N matrix of Bytes,
// the BytesList must contain M*N bytes, with M rows of N contiguous values
// each.  That is, the BytesList value must store the matrix as:
//     .... row 0 .... .... row 1 .... // ...........  // ... row M-1 ....
//
// An Example for a movie recommendation application:
//   features {
//     feature {
//       key: "age"
//       value { float_list {
//         value: 29.0
//       }}
//     }
//     feature {
//       key: "movie"
//       value { bytes_list {
//         value: "The Shawshank Redemption"
//         value: "Fight Club"
//       }}
//     }
//     feature {
//       key: "movie_ratings"
//       value { float_list {
//         value: 9.0
//         value: 9.7
//       }}
//     }
//     feature {
//       key: "suggestion"
//       value { bytes_list {
//         value: "Inception"
//       }}
//     }
//     # Note that this feature exists to be used as a label in training.
//     # E.g., if training a logistic regression model to predict purchase
//     # probability in our learning tool we would set the label feature to
//     # "suggestion_purchased".
//     feature {
//       key: "suggestion_purchased"
//       value { float_list {
//         value: 1.0
//       }}
//     }
//     # Similar to "suggestion_purchased" above this feature exists to be used
//     # as a label in training.
//     # E.g., if training a linear regression model to predict purchase
//     # price in our learning tool we would set the label feature to
//     # "purchase_price".
//     feature {
//       key: "purchase_price"
//       value { float_list {
//         value: 9.99
//       }}
//     }
//  }
//
// A conformant Example data set obeys the following conventions:
//   - If a Feature K exists in one example with data type T, it must be of
//       type T in all other examples when present. It may be omitted.
//   - The number of instances of Feature K list data may vary across examples,
//       depending on the requirements of the model.
//   - If a Feature K doesn't exist in an example, a K-specific default will be
//       used, if configured.
//   - If a Feature K exists in an example but contains no items, the intent
//       is considered to be an empty tensor and no default will be used.

message Example {
  Features features = 1;
};

// A SequenceExample is an Example representing one or more sequences, and
// some context.  The context contains features which apply to the entire
// example. The feature_lists contain a key, value map where each key is
// associated with a repeated set of Features (a FeatureList).
// A FeatureList thus represents the values of a feature identified by its key
// over time / frames.
//
// Below is a SequenceExample for a movie recommendation application recording a
// sequence of ratings by a user. The time-independent features ("locale",
// "age", "favorites") describing the user are part of the context. The sequence
// of movies the user rated are part of the feature_lists. For each movie in the
// sequence we have information on its name and actors and the user's rating.
// This information is recorded in three separate feature_list(s).
// In the example below there are only two movies. All three feature_list(s),
// namely "movie_ratings", "movie_names", and "actors" have a feature value for
// both movies. Note, that "actors" is itself a bytes_list with multiple
// strings per movie.
//
// context: {
//   feature: {
//     key  : "locale"
//     value: {
//       bytes_list: {
//         value: [ "pt_BR" ]
//       }
//     }
//   }
//   feature: {
//     key  : "age"
//     value: {
//       float_list: {
//         value: [ 19.0 ]
//       }
//     }
//   }
//   feature: {
//     key  : "favorites"
//     value: {
//       bytes_list: {
//         value: [ "Majesty Rose", "Savannah Outen", "One Direction" ]
//       }
//     }
//   }
// }
// feature_lists: {
//   feature_list: {
//     key  : "movie_ratings"
//     value: {
//       feature: {
//         float_list: {
//           value: [ 4.5 ]
//         }
//       }
//       feature: {
//         float_list: {
//           value: [ 5.0 ]
//         }
//       }
//     }
//   }
//   feature_list: {
//     key  : "movie_names"
//     value: {
//       feature: {
//         bytes_list: {
//           value: [ "The Shawshank Redemption" ]
//         }
//       }
//       feature: {
//         bytes_list: {
//           value: [ "Fight Club" ]
//         }
//       }
//     }
//   }
//   feature_list: {
//     key  : "actors"
//     value: {
//       feature: {
//         bytes_list: {
//           value: [ "Tim Robbins", "Morgan Freeman" ]
//         }
//       }
//       feature: {
//         bytes_list: {
//           value: [ "Brad Pitt", "Edward Norton", "Helena Bonham Carter" ]
//         }
//       }
//     }
//   }
// }
//
// A conformant SequenceExample data set obeys the following conventions:
//
// Context:
//   - All conformant context features K must obey the same conventions as
//     a conformant Example's features (see above).
// Feature lists:
//   - A FeatureList L may be missing in an example; it is up to the
//     parser configuration to determine if this is allowed or considered
//     an empty list (zero length).
//   - If a FeatureList L exists, it may be empty (zero length).
//   - If a FeatureList L is non-empty, all features within the FeatureList
//     must have the same data type T. Even across SequenceExamples, the type T
//     of the FeatureList identified by the same key must be the same. An entry
//     without any values may serve as an empty feature.
//   - If a FeatureList L is non-empty, it is up to the parser configuration
//     to determine if all features within the FeatureList must
//     have the same size.  The same holds for this FeatureList across multiple
//     examples.
//
// Examples of conformant and non-conformant examples' FeatureLists:
//
// Conformant FeatureLists:
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { float_list: { value: [ 5.0 ] } } }
//    } }
//
// Non-conformant FeatureLists (mismatched types):
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { int64_list: { value: [ 5 ] } } }
//    } }
//
// Conditionally conformant FeatureLists, the parser configuration determines
// if the feature sizes must match:
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { float_list: { value: [ 5.0, 6.0 ] } } }
//    } }
//
// Conformant pair of SequenceExample
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { float_list: { value: [ 5.0 ] } } }
//    } }
// and:
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { float_list: { value: [ 5.0 ] } }
//               feature: { float_list: { value: [ 2.0 ] } } }
//    } }
//
// Conformant pair of SequenceExample
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { float_list: { value: [ 5.0 ] } } }
//    } }
// and:
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { }
//    } }
//
// Conditionally conformant pair of SequenceExample, the parser configuration
// determines if the second feature_lists is consistent (zero-length) or
// invalid (missing "movie_ratings"):
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { float_list: { value: [ 5.0 ] } } }
//    } }
// and:
//    feature_lists: { }
//
// Non-conformant pair of SequenceExample (mismatched types)
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { float_list: { value: [ 5.0 ] } } }
//    } }
// and:
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { int64_list: { value: [ 4 ] } }
//               feature: { int64_list: { value: [ 5 ] } }
//               feature: { int64_list: { value: [ 2 ] } } }
//    } }
//
// Conditionally conformant pair of SequenceExample; the parser configuration
// determines if the feature sizes must match:
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.5 ] } }
//               feature: { float_list: { value: [ 5.0 ] } } }
//    } }
// and:
//    feature_lists: { feature_list: {
//      key: "movie_ratings"
//      value: { feature: { float_list: { value: [ 4.0 ] } }
//               feature: { float_list: { value: [ 5.0, 3.0 ] } }
//    } }

message SequenceExample {
  Features context = 1;
  FeatureLists feature_lists = 2;
};


 

feature.proto

syntax = "proto3";
option cc_enable_arenas = true;
option java_outer_classname = "FeatureProtos";
option java_multiple_files = true;
option java_package = "org.tensorflow.example";

package tensorflow;

// Containers to hold repeated fundamental values.
message BytesList {
  repeated bytes value = 1;
}
message FloatList {
  repeated float value = 1 [packed = true];
}
message Int64List {
  repeated int64 value = 1 [packed = true];
}

// Containers for non-sequential data.
message Feature {
  // Each feature can be exactly one kind.
  oneof kind {
    BytesList bytes_list = 1;
    FloatList float_list = 2;
    Int64List int64_list = 3;
  }
};

message Features {
  // Map from feature name to feature.
  map<string, Feature> feature = 1;
};

// Containers for sequential data.
//
// A FeatureList contains lists of Features.  These may hold zero or more
// Feature values.
//
// FeatureLists are organized into categories by name.  The FeatureLists message
// contains the mapping from name to FeatureList.
//
message FeatureList {
  repeated Feature feature = 1;
};

message FeatureLists {
  // Map from feature name to feature list.
  map<string, FeatureList> feature_list = 1;
};
 

看看這兩個proto就能YY出他們的關係大概可以表示成example{features:{‘feature1’:xxx, ‘feature2’:yyy, …, ‘featuren’:23333}}。

捋清楚關係後，就可以想一個問題了。我的資料是一張張的圖片，那最原始的特徵就是畫素，所以可以嘗試把圖的畫素看成是位元組快然後序列化成字串來表示一張圖。所以就有了下面的程式碼：

#轉換成records
def conver_to_tfrecords(data_set, label_set, name):
    #proto裡提到的int64list
    def _int64_feature(value):
        return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

    #proto裡提到的byteslist
    def _bytes_feature(value):
        return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

    print('正在轉換成tfrecords', name)
    #例項化tfrecord的writer，說白了就是用來寫檔案的物件
    writer = tf.python_io.TFRecordWriter(name)
    num_examples = len(data_set)
    for index in range(num_examples):
        image = data_set[index]
        height = image.shape[0]
        width = image.shape[1]
        #把畫素轉成字串
        image_raw = image.tostring()
        label = label_set[index]
        #把onehot後的label轉成字串
        label_raw = label_to_one_hot(label).tostring()
        #構建example，每個examp由圖的高，寬，label， 畫素組成
        example = tf.train.Example(features=tf.train.Features(feature={
            'height': _int64_feature(height),
            'width': _int64_feature(width),
            'label_raw': _bytes_feature(label_raw),
            'image_raw': _bytes_feature(image_raw)}))
        #把example序列化成字串並寫到名字為name的檔案中
        writer.write(example.SerializeToString())
    writer.close()
    print('轉換完畢!')

這個時候，資料已經全部寫到tfrecords檔案中了。那接下來要乾的事情就是用佇列讀資料。用佇列讀資料的好處是可以主執行緒用來訓練模型，子執行緒用來從佇列裡拿資料，寫過程式碼的都知道這樣做有啥好處。。。

讀資料

那要用佇列，首先肯定是要建立個佇列，那建立佇列我這裡用的tf.train.string_input_producer，因為我的影象和label都序列化成了字串。
那有佇列之後我肯定是要從佇列裡拿資料啦！！所以就會有下面的程式碼：

def read_and_decode(filename_queue, img_height=IMG_HEIGHT, img_width=IMG_WIDTH, chars_num=CHAR_NUM, classes_num=len(CHAR_SET)):
    #例項化tfrecord的reader，用來讀資料的物件
    reader = tf.TFRecordReader()
    #從建立好的字串佇列中讀資料，現在還是二進位制的字串資料
    _, serialized_example = reader.read(filename_queue)
    #解析二進位制資料，PS:我寫tfrecords的時候圖和label是string，所以我解析的時候也要是string
    features = tf.parse_single_example(
        serialized_example,
        features={
          'image_raw': tf.FixedLenFeature([], tf.string),
          'label_raw': tf.FixedLenFeature([], tf.string),
      })
   #這裡相當於把string轉成float32，因為畫素值不可能是字元嘛~~~
    image = tf.decode_raw(features['image_raw'], tf.float32)
    image.set_shape([img_height * img_width])
    image = tf.cast(image, tf.float32) * (1.0 / 255)-0.5
    reshape_image = tf.reshape(image, [img_height, img_width, 1])
    label = tf.decode_raw(features['label_raw'], tf.float32)
    label.set_shape([chars_num * classes_num])
    reshape_label = tf.reshape(label, [chars_num, classes_num])
    return tf.cast(reshape_image, tf.float32), tf.cast(reshape_label, tf.float32)


def inputs(train, batch_size, epoch):
    filename = os.path.join('./', TRAIN_RECORDS_NAME if train else TRAIN_VALIDATE_NAME)

    with tf.name_scope('input'):
        filename_queue = tf.train.string_input_producer([filename], num_epochs=epoch)
        image, label = read_and_decode(filename_queue)
        #如果現在是訓練，那我會打亂順序再一個一個batch的拿資料，如果是驗證那我就按順序那batch
        if train:
            images, sparse_labels = tf.train.shuffle_batch([image, label], batch_size=batch_size, num_threads=6, capacity=2000 + 3 * batch_size, min_after_dequeue=2000)
        else:
            images, sparse_labels = tf.train.batch([image, label], batch_size=batch_size, num_threads=6, capacity=2000 + 3 * batch_size)

    return images, sparse_labels

程式碼的意思已經寫成註釋了，不過要注意的是，tensorflow要執行graph需要session，所以你需要用session來run你的inputs才會真正的讓佇列裡真正的有資料。至於怎麼run起這個佇列，後面再來嗶嗶。今天就到這裡。下一篇估計會記錄一下整個CNN怎麼搭的。