get .gz ofo 模型 world data end class pwd

基本分類

基本分類（Basic classification）：https://www.tensorflow.org/tutorials/keras/basic_classification

Fashion MNIST數據集

• 經典 MNIST 數據集（常用作計算機視覺機器學習程序的“Hello, World”入門數據集）的簡易替換
• 包含訓練數據60000個，測試數據10000個，每個圖片是28x28像素的灰度圖像，涵蓋10個類別
• TensorFlow：https://www.tensorflow.org/api_docs/python/tf/keras/datasets/fashion_mnist
• GitHub：https://github.com/zalandoresearch/fashion-mnist

tf.keras

是一種用於在TensorFlow中構建和訓練模型的高階API：https://www.tensorflow.org/api_docs/python/tf/keras/

示例

腳本內容

xxx

運行結果

xxx

問題處理

問題1：執行fashion_mnist.load_data()失敗

錯誤提示
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz

......
Exception: URL fetch failure on https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz: None -- [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

處理方法1

選擇一個鏈接，

• https://github.com/zalandoresearch/fashion-mnist/tree/master/data/fashion
• https://storage.googleapis.com/tensorflow/tf-keras-datasets/

手工下載下面四個文件，並存放在“~/.keras/datasets”下的fashion-mnist目錄。

• train-labels-idx1-ubyte.gz
• train-images-idx3-ubyte.gz
• t10k-labels-idx1-ubyte.gz
• t10k-images-idx3-ubyte.gz

guowli@5CG450158J MINGW64 ~/.keras/datasets
$ pwd
/c/Users/guowli/.keras/datasets

guowli@5CG450158J MINGW64 ~/.keras/datasets
$ ls -l
total 0
drwxr-xr-x 1 guowli 1049089 0 Mar 27 14:44 fashion-mnist/

guowli@5CG450158J MINGW64 ~/.keras/datasets
$ ls -l fashion-mnist/
total 30164
-rw-r--r-- 1 guowli 1049089  4422102 Mar 27 15:47 t10k-images-idx3-ubyte.gz
-rw-r--r-- 1 guowli 1049089     5148 Mar 27 15:47 t10k-labels-idx1-ubyte.gz
-rw-r--r-- 1 guowli 1049089 26421880 Mar 27 15:47 train-images-idx3-ubyte.gz
-rw-r--r-- 1 guowli 1049089    29515 Mar 27 15:47 train-labels-idx1-ubyte.gz

處理方法2

手工下載文件，存放在指定目錄。
改寫“tensorflow\python\keras\datasets\fashion_mnist.py”定義的load_data()函數。

from tensorflow.python.keras.utils import get_file
import numpy as np
import pathlib
import gzip


def load_data():  # 改寫“tensorflow\python\keras\datasets\fashion_mnist.py”定義的load_data()函數
    base = "file:///" + str(pathlib.Path.cwd()) + "\\"  # 當前目錄

    files = [
        ‘train-labels-idx1-ubyte.gz‘, ‘train-images-idx3-ubyte.gz‘,
        ‘t10k-labels-idx1-ubyte.gz‘, ‘t10k-images-idx3-ubyte.gz‘
    ]

    paths = []
    for fname in files:
        paths.append(get_file(fname, origin=base + fname))

    with gzip.open(paths[0], ‘rb‘) as lbpath:
        y_train = np.frombuffer(lbpath.read(), np.uint8, offset=8)

    with gzip.open(paths[1], ‘rb‘) as imgpath:
        x_train = np.frombuffer(
            imgpath.read(), np.uint8, offset=16).reshape(len(y_train), 28, 28)

    with gzip.open(paths[2], ‘rb‘) as lbpath:
        y_test = np.frombuffer(lbpath.read(), np.uint8, offset=8)

    with gzip.open(paths[3], ‘rb‘) as imgpath:
        x_test = np.frombuffer(
            imgpath.read(), np.uint8, offset=16).reshape(len(y_test), 28, 28)

    return (x_train, y_train), (x_test, y_test)


(train_images, train_labels), (test_images, test_labels) = load_data()

問題2：使用gzip.open()打開.gz文件失敗

錯誤提示

“OSError: Not a gzipped file (b‘\n\n‘)”

處理方法

對於損壞的、不完整的.gz文件，zip.open()將無法打開。檢查.gz文件是否完整無損。

參考信息

https://github.com/tensorflow/tensorflow/issues/170

AI - TensorFlow - 示例01：基本分類