論文：Remote Sensing Image Scene Classification 程式碼執行過程各種坑
論文：Remote Sensing Image Scene Classification 程式碼
Md. Arafat Hussain, Emon Kumar Dey*
github上的
Remote Sensing Image Scene Classification Using Deep Learning
Here we classified scene images of NWPU-RESISC45 dataset
The dataset can be downloaded from http://www.escience.cn/people/JunweiHan/NWPU-RESISC45.html
We applied Convolutional Neural Network on this dataset to classifiy images. We trained the dataset from scratch. Also appied transfer learning using pre-trained VGG16 abd ResNet50.
We used training_from_scratch.py to train the dataset from scratch. We used python 2.7, keras 2.05 and Theano backend 1.0,OpenCV 3.30.
Applied transfer learning using VGG16_Transfer_Learning.py and RestNet50_Transfer_Learning.py. Here we used python 3.5, keras, 2.05 using Tensorfloe backend 1.0 and openCV 3.30
/usr/bin/python3.6 /home/project/rsisc/training_from_scratch.py
Traceback (most recent call last):
File "/home/project/rsisc/training_from_scratch.py", line 10, in <module>
import os,cv2
ModuleNotFoundError: No module named 'cv2'
Process finished with exit code 1
Collecting opencv-python
Downloading https://files.pythonhosted.org/packages/18/7f/c836c44ab30074a8486e30f8ea6adc8e6ac02332851ab6cc069e2ac35b84/opencv_python-3.4.3.18-cp36-cp36m-manylinux1_x86_64.whl (25.0MB)
100% |████████████████████████████████| 25.0MB 135kB/s
Requirement already satisfied: numpy>=1.11.3 in /home/gden/anaconda3/lib/python3.6/site-packages (from opencv-python) (1.14.3)
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: opencv-python
Could not install packages due to an EnvironmentError: [Errno 13] 許可權不夠: '/home/gden/anaconda3/lib/python3.6/site-packages/cv2'
Consider using the `--user` option or check the permissions.
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[email protected]:/media/gden/6C72D37472D3420E$ pip install --upgrade pip
Collecting pip
Using cached https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: pip
Found existing installation: pip 10.0.1
Uninstalling pip-10.0.1:
Could not install packages due to an EnvironmentError: [Errno 13] 許可權不夠: '/home/gden/anaconda3/bin/pip'
Consider using the `--user` option or check the permissions.
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[email protected]:/media/gden/6C72D37472D3420E$ sudo pip install opencv-python --user
[sudo] gden 的密碼：
sudo: pip：找不到命令
[email protected]:/media/gden/6C72D37472D3420E$ pip install opencv-python --user
Collecting opencv-python
Using cached https://files.pythonhosted.org/packages/18/7f/c836c44ab30074a8486e30f8ea6adc8e6ac02332851ab6cc069e2ac35b84/opencv_python-3.4.3.18-cp36-cp36m-manylinux1_x86_64.whl
Requirement already satisfied: numpy>=1.11.3 in /home/gden/anaconda3/lib/python3.6/site-packages (from opencv-python) (1.14.3)
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: opencv-python
Successfully installed opencv-python-3.4.3.18
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
2\
/usr/bin/python3.6 /home/project/rsisc/training_from_scratch.py
ImportError: numpy.core.multiarray failed to import
Traceback (most recent call last):
File "/home/project/rsisc/training_from_scratch.py", line 10, in <module>
import os,cv2
File "/home/gden/.local/lib/python3.6/site-packages/cv2/__init__.py", line 3, in <module>
from .cv2 import *
ImportError: numpy.core.multiarray failed to import
Process finished with exit code 1
需要更新numpy版本：
[email protected]:/media/gden/6C72D37472D3420E$ pip install -U numpy
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError("bad handshake: SysCallError(104, 'ECONNRESET')",),)': /simple/numpy/
Collecting numpy
Using cached https://files.pythonhosted.org/packages/22/02/bae88c4aaea4256d890adbf3f7cf33e59a443f9985cf91cd08a35656676a/numpy-1.15.2-cp36-cp36m-manylinux1_x86_64.whl
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: numpy
Found existing installation: numpy 1.14.3
Uninstalling numpy-1.14.3:
Could not install packages due to an EnvironmentError: [Errno 13] 許可權不夠: 'matlib.py'
Consider using the `--user` option or check the permissions.
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[email protected]:/media/gden/6C72D37472D3420E$ pip install -U numpy --user
Collecting numpy
Using cached https://files.pythonhosted.org/packages/22/02/bae88c4aaea4256d890adbf3f7cf33e59a443f9985cf91cd08a35656676a/numpy-1.15.2-cp36-cp36m-manylinux1_x86_64.whl
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: numpy
Successfully installed numpy-1.15.2
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
3
/usr/bin/python3.6 /home/project/rsisc/training_from_scratch.py
Traceback (most recent call last):
File "/home/project/rsisc/training_from_scratch.py", line 12, in <module>
import matplotlib.pyplot as plt
ModuleNotFoundError: No module named 'matplotlib'
Process finished with exit code 1
折騰半天：
/home/gden/.conda/envs/tensorflow-gpu/bin/python /home/project/rsisc/training_from_scratch.py
Traceback (most recent call last):
File "/home/project/rsisc/training_from_scratch.py", line 12, in <module>
import matplotlib.pyplot as plt
File "/home/gden/.local/lib/python3.6/site-packages/matplotlib/__init__.py", line 141, in <module>
from . import cbook, rcsetup
File "/home/gden/.local/lib/python3.6/site-packages/matplotlib/rcsetup.py", line 25, in <module>
from matplotlib.fontconfig_pattern import parse_fontconfig_pattern
File "/home/gden/.local/lib/python3.6/site-packages/matplotlib/fontconfig_pattern.py", line 19, in <module>
from pyparsing import (Literal, ZeroOrMore, Optional, Regex, StringEnd,
ModuleNotFoundError: No module named 'pyparsing'
Process finished with exit code 1
後來執行：
conda list發現tensorflow沒有配好，重新安裝，參照上面的一篇博文，注意細節，要把對應的包安裝到tensorflow開啟的環境下。
最後終於搞定，但提示 58行程式碼 記憶體錯誤。折騰了半天發現是資料集太大，不能一次調取，需要分批。換了小資料集繼續進行。一切ok。
然後執行後發現380行有錯誤；
test_image = cv2.imread('data/runway/runway08.tif')
test_image=cv2.cvtColor(test_image, cv2.COLOR_BGR2GRAY) #380行原來上一行的路徑圖片不存在，進行修改繼續執行
rennet50執行出錯：
/home/gden/.conda/envs/tensorflow/bin/python3.6 /home/project/rsisc/resnet50.py
Using TensorFlow backend.
Traceback (most recent call last):
File "/home/project/rsisc/resnet50.py", line 31, in <module>
from keras.applications.imagenet_utils import _obtain_input_shape
ImportError: cannot import name '_obtain_input_shape'
Process finished with exit code 1
/home/gden/.conda/envs/tensorflow/bin/python3.6 /home/project/rsisc/resnet50.py
Using TensorFlow backend.
Traceback (most recent call last):
File "/home/project/rsisc/resnet50.py", line 277, in <module>
model = ResNet50(include_top=True, weights='imagenet') #引數個數發生變化
File "/home/project/rsisc/resnet50.py", line 181, in ResNet50
include_top=include_top)
TypeError: _obtain_input_shape() got an unexpected keyword argument 'include_top'
Process finished with exit code 1
在執行其他開源keras專案時，遇到了一些問題：
1、在匯入_obtain_input_shape時
from keras.applications.imagenet_utils import _obtain_input_shape
1
出現錯誤如下：
ImportError: cannot import name '_obtain_input_shape'
1
原因是在keras 2.2.2中，keras.applications.imagenet_utils模組不再有_obtain_input_shape方法。解決方法：
將匯入語句修改如下
1from keras_applications.imagenet_utils import _obtain_input_shape
重點：使用keras_applications代替keras.applications
2、在使用_obtain_input_shape方法時出現錯誤：
_obtain_input_shape() got an unexpected keyword argument 'include_top'
1
原因是我們的呼叫中使用了
input_shape = _obtain_input_shape(input_shape,
default_size=224,
min_size=32,
data_format=K.image_data_format(),
include_top=include_top or weights)
1
2
3
4
5
而在keras 2.2.2中函式 _obtain_input_shape() 的形式為：
def _obtain_input_shape(input_shape,
default_size,
min_size,
data_format,
require_flatten,
weights=None):
自然沒有include_top屬性。因此把呼叫中的include_top改為require_flatten即可
---------------------
作者：sssaltyfish
來源：CSDN
原文：https://blog.csdn.net/weixin_43344725/article/details/83187692
版權宣告：本文為博主原創文章，轉載請附上博文連結！
File "/home/project/rsisc/resnet50.py", line 280, in <module>
img = image.load_img(img_path, target_size=(224, 224))
File "/home/gden/.local/lib/python3.6/site-packages/keras_preprocessing/image.py", line 498, in load_img
img = pil_image.open(path)
File "/home/gden/.conda/envs/tensorflow/lib/python3.6/site-packages/PIL/Image.py", line 2477, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'elephant.jpg'
Process finished with exit code 1
img_path = 'elephant.jpg'這一行的檔案不存在，所以放一個存在的在路徑下就行了。
中間用的ubuntu命令：
解壓rar檔案，安裝rar工具：sudo apt-get install rar 解除安裝 sudo apt-get remove rar 解壓 rar x 壓縮檔名
解壓zip檔案 unzip 壓縮檔名
許可權不夠的情況 sudo 命令 最後加--user
mv 資料夾 新資料夾 作用移動資料夾，重新命名
cp -r 資料夾 路徑 複製資料夾中所有檔案到新路徑
rm -rf 檔案或資料夾 刪除命令
下面文章可以參考：https://blog.csdn.net/lilong117194/article/details/79138265
我的問題是：
在使用Jupyter Notebook跑tensorflow程式時，出現報錯ImportError: No module named 'matplotlib'，然而在spyder（非tensorflow環境）下執行import matplotlib.pyplot as plt就沒有問題。然後在網上查了很多資料，大部分都是說：
（1）用終端“sudo apt-get install python-matplotlib ”，安裝matplotlib包
（2）或者是在tensorflow環境下進行“sudo apt-get install python-matplotlib ”包的安裝。
以上我都試了，然後開啟Jupyter Notebook跑tensorflow程式還是報錯。
最後經過多次嘗試，終於搞定。
首先我的系統是ubuntu 14.04 的，然後安裝的是Anaconda 2和python2.7，以上是一些基礎資訊。這裡安裝matplotlib包其實是這樣的：
（1）首先進入tensorflow的環境：$ source activate tensorflow
（2）通過命令安裝matplotlib包：conda install matplotlib
（3）然後就是根據提示選擇“y”，確認安裝
再次執行tensorflow程式中的import matplotlib.pyplot as plt 就沒問題了。
這裡要說明的是：
（1）Anaconda
是一個整合許多第三方科學計算庫的 Python 科學計算環境，用 conda 作為自己的包管理工具，同時具有自己的計算環境，類似 Virtualenv。
（2）Conda
Conda 是一個開源的軟體包管理系統和環境管理系統，用於安裝多個版本的軟體包及其依賴關係，並在它們之間輕鬆切換。
常用命令：
conda list 列出當前 conda 環境所連結的軟體包
conda create 建立一個 conda 環境，名稱為 tf
conda create -n 環境名 -c 映象源
建立conda執行環境 ：
Create a conda environment called tensorflow:
Python 2.7
$ conda create -n tensorflow python=2.7
Python 3.4
$ conda create -n tensorflow python=3.4
Python 3.5
$ conda create -n tensorflow python=3.5
1
2
3
4
5
6
7
8
啟用\關閉conda執行環境：
source activate tensorflow
source deactivate tensorflow
1
2
參考：
https://stackoverflow.com/questions/42356396/importerror-no-module-named-matplotlib-using-anaconda-tensorflow-environme
http://blog.csdn.net/cs_leebo/article/details/64444243
---------------------
作者：lilong117194
來源：CSDN
原文：https://blog.csdn.net/lilong117194/article/details/79138265
版權宣告：本文為博主原創文章，轉載請附上博文連結！
進入mobaXterm工具，輸入使用者名稱登陸：
然後輸入：
Downloads/pycharm-community-2017.2/bin/pycharm.sh
啟動工具。
Remote Sensing Image Scene Classification Using Deep Learning
Here we classified scene images of NWPU-RESISC45 dataset
The dataset can be downloaded from http://www.escience.cn/people/JunweiHan/NWPU-RESISC45.html
We applied Convolutional Neural Network on this dataset to classifiy images. We trained the dataset from scratch. Also appied transfer learning using pre-trained VGG16 abd ResNet50.
We used training_from_scratch.py to train the dataset from scratch. We used python 2.7, keras 2.05 and Theano backend 1.0,OpenCV 3.30.
Applied transfer learning using VGG16_Transfer_Learning.py and RestNet50_Transfer_Learning.py. Here we used python 3.5, keras, 2.05 using Tensorfloe backend 1.0 and openCV 3.30
執行程式：
[email protected]:~/PycharmProjects/guo/rsisc$ python training_from_scratch.py
Traceback (most recent call last):
File "training_from_scratch.py", line 8, in <module>
import os,cv2
ModuleNotFoundError: No module named 'cv2'
在安裝opevncv時會出現 ImportError: No module named cv2 的錯誤，找不到cv2的包。
這時候安裝擴充套件包即可：
pip install opencv-python
結果：
Collecting opencv-python
Downloading https://files.pythonhosted.org/packages/18/7f/c836c44ab30074a8486e30f8ea6adc8e6ac02332851ab6cc069e2ac35b84/opencv_python-3.4.3.18-cp36-cp36m-manylinux1_x86_64.whl (25.0MB)
100% |████████████████████████████████| 25.0MB 50kB/s
Requirement already satisfied: numpy>=1.11.3 in /home/gis/anaconda3/lib/python3.6/site-packages (from opencv-python) (1.14.3)
Installing collected packages: opencv-python
Successfully installed opencv-python-3.4.3.18
然後重新執行：
[email protected]:~/PycharmProjects/guo/rsisc$ python training_from_scratch.py
/home/gis/anaconda3/lib/python3.6/site-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
Traceback (most recent call last):
File "training_from_scratch.py", line 15, in <module>
from keras import backend as K
ModuleNotFoundError: No module named 'keras'
缺少包 keras
執行：pip3 install --upgrade keras
Collecting keras
Downloading https://files.pythonhosted.org/packages/34/7d/b1dedde8af99bd82f20ed7e9697aac0597de3049b1f786aa2aac3b9bd4da/Keras-2.2.2-py2.py3-none-any.whl (299kB)
100% |████████████████████████████████| 307kB 27kB/s
Collecting keras-applications==1.0.4 (from keras)
Downloading https://files.pythonhosted.org/packages/54/90/8f327deaa37a71caddb59b7b4aaa9d4b3e90c0e76f8c2d1572005278ddc5/Keras_Applications-1.0.4-py2.py3-none-any.whl (43kB)
100% |████████████████████████████████| 51kB 7.6kB/s
Requirement already satisfied, skipping upgrade: numpy>=1.9.1 in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (1.14.3)
Requirement already satisfied, skipping upgrade: h5py in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (2.7.1)
Requirement already satisfied, skipping upgrade: scipy>=0.14 in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (1.1.0)
Collecting keras-preprocessing==1.0.2 (from keras)
Downloading https://files.pythonhosted.org/packages/71/26/1e778ebd737032749824d5cba7dbd3b0cf9234b87ab5ec79f5f0403ca7e9/Keras_Preprocessing-1.0.2-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: pyyaml in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (3.12)
Requirement already satisfied, skipping upgrade: six>=1.9.0 in /home/gis/anaconda3/lib/python3.6/site-packages (from keras) (1.11.0)
Installing collected packages: keras-applications, keras-preprocessing, keras
Successfully installed keras-2.2.2 keras-applications-1.0.4 keras-preprocessing-1.0.2
重新執行：[email protected]:~/PycharmProjects/guo/rsisc$ python training_from_scratch.py
一切ok
[email protected]:~/PycharmProjects/guo/rsisc$ python training_from_scratch.py
/home/gis/anaconda3/lib/python3.6/site-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
/home/gis/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Loaded the images of dataset-meadow
Loaded the images of dataset-circular_farmland
。。。。。。。。。
二、
# MARTA-GAN
This is the code for [MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification](https://arxiv.org/abs/1612.08879). An multiple-layer feature-matching generative adversarial networks (MARTA GANs) to learn a representation using only unlabeled data.
## Prepare data
Download and unzip dataset from [BaiDuYun](https://pan.baidu.com/s/1i5zQNdj) or [Google Drive](https://drive.google.com/open?id=0B1Evui8Soh85ZXM3cDNvbGdOamc).
## Dependencies
NVIDIA GPU + CUDA CuDNN (CPU mode and CUDA without CuDNN mode are also available but significantly slower)
- tensorflow
- tensorlayer
- sklearn
## Usage
Training GAN
```
python train_marta_gan.py
```
Extract features
```
python extract_feature.py
```
Training SVM
```
python train_svm.py
```
## Citation
If you find this code useful for your research, please cite:
```
@article{lin2017marta,
title={MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification},
author={Lin, Daoyu and Fu, Kun and Wang, Yang and Xu, Guangluan and Sun, Xian},
journal={IEEE Geoscience and Remote Sensing Letters},
year={2017},
publisher={IEEE}
}
```
首先執行程式：
[email protected]:~/PycharmProjects/guo/MARTA-GAN-master$ python train_marta_gan.py
File "train_marta_gan.py", line 112
print net_g_name, net_d_name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(net_g_name, net_d_name)?
112 print 少括號，3.6版本下
[email protected]:~/PycharmProjects/guo/MARTA-GAN-master$ python train_marta_gan.py
File "train_marta_gan.py", line 142
print sample_images.shape
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(sample_images.shape)?
142，186行一樣少括號，改完後在執行：
[email protected]:~/PycharmProjects/guo/MARTA-GAN-master$ python train_marta_gan.py
/home/gis/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "train_marta_gan.py", line 8, in <module>
import tensorlayer as tl
ModuleNotFoundError: No module named 'tensorlayer'
提示少tensorlayer包：TensorLayer 是為研究人員和工程師設計的一款基於Google TensorFlow開發的深度學習與強化學習庫。 它提供高級別的（Higher-Level）深度學習API，這樣不僅可以加快研究人員的實驗速度，也能夠減少工程師在實際開發當中的重複工作。 TensorLayer非常易於修改和擴充套件，這使它可以同時用於機器學習的研究與應用。 此外，TensorLayer 提供了大量示例和教程來幫助初學者理解深度學習，並提供大量的官方例子程式方便開發者快速找到適合自己專案的例子。
TensorLayer 是建立在 Google TensorFlow 上的深度學習(Deep Learning)與增強學習(Reinforcement Learning) 軟體庫。它的優勢在於既能提供高階 API 以方便開發者快速搭建網路模型，也能保證與 TensorFlow 的全透明銜接。原始碼簡潔，非常容易修改與拓展，同時適合於演算法研究人員和產品開發者，加快新演算法產品化。 官方例子涵蓋 Stacked Denoising Autoencoder, Dropout, CNN, LSTM, Word Embedding, Language Modelling, Machine Translation, Deep Reinforcement Learning等等。官方網站 不僅僅描述瞭如何使用該API，它還包含所有官方例子的演算法教程。
個人強烈推薦 TensorLayer 的原因是它不僅僅是一個功能齊全的工具，它與 TensorFlow 全透明的銜接方式是 Keras、Tflearn 沒法相提並論的；對研究者而言，可以非常方便地修改訓練的過程，而不會像其他庫用簡單的一行 fit() 搞定，根本沒法讓人修改內部運作，然而對研究者而言最重要的是它的原始碼真的寫得太棒了，我覺得它是我見過最簡潔的python程式碼，我們的演算法可以很容易拓展進去，然後和它已有的功能一起使用！！！ 對普通開發者而言，它的好處是例子多，例子的程式碼風格保持一致，不像TensorFlow官方程式碼那樣每個例子的程式碼風格完全不同。。一致的程式碼風格有助於開發人員快速找到合適的模型來開發產品。
最後，它的開發者是帝國理工計算機系的博士，陸陸續續有帝國的大神支撐，不像其他框架只是個人在開發。我們可不希望辛辛苦苦學一個框架，然後這個框架最後不再維護了。
此外，值得注意的是 TuneLayer 的前身是 TensorLayer 它是為科學家和工程師而設計的一款基於Google TensorFlow開發的深度學習與增強學習庫。使用 TLayer 可以大大加快演算法實驗和產品開發速度。它非常容易拓展和修改，並提供大量的官方例子程式，方便開發者快速找到適合自己專案的例子。
官方文件 不僅僅描述如何使用 TuneLayer API，還包含了大量的教程，覆蓋不同的神經網路型別、深度增強學習和自然語言處理等等。不過，與其它基於TensorFlow開發的傻瓜式API不同，TuneLayer需要使用者有基本的神經網路知識。瞭解TensorFlow的基礎，可以讓用非常熟練地使用它。
設計理念
TuneLayer 的發展需要和 TensorFlow 緊密結合，因此 TLayer 的設計具有很強的可塑性，它追隨如下的設計理念：
Transparency（透明）：不會去隱藏 TensorFlow，而是儘可能地依賴 TensorFlow 的方法，追隨 TensorFlow 的慣例。不會去隱藏訓練過程，所有迭代、初始化都可以被使用者管理。
Tensor （張量）：神經網路通過高緯度資料表達。
TPU（張量處理單元）：張量處理單元是 Google 設計的專門執行機器學習的定製化 ASIC，除了 TPU 我們將不斷相容更多的機器學習 ASIC。
Distribution（分散式）：分散式機器學習是 TensorFlow 的基本功能，TLayer 在 TensorFlow 的基礎上結合 Spark ，讓分散式計算更好地服務於機器學習。
Compatibility（相容性）：網路被抽象為規則函式、損失函式和每層輸出。容易與其它 TensorFlow 庫結合。
Simplicity（簡化）：容易使用，容易拓展與修改，加快研究成果的產品化進度。
High-Speed（高速）：在使用GPU時，執行速度和純TensorFlow程式碼一樣，不會因為簡化程式碼而犧牲效能。
安裝 tensorlayer
2018年08月27日 13:45:47 luoganttcc 閱讀數：59更多
個人分類： tensorlayer
版權宣告：本文為博主原創文章，未經博主允許不得轉載。 https://blog.csdn.net/luoganttcc/article/details/82111149
pip install --user tensorlayer
[email protected]:~/PycharmProjects/guo/MARTA-GAN-master$ pip install --user tensorlayer
Collecting tensorlayer
Using cached https://files.pythonhosted.org/packages/c4/e2/736458723564163cfd87e3a9719a9fdece9011429bf556fb910d3691352e/tensorlayer-1.10.1-py2.py3-none-any.whl
Collecting scikit-image<0.15,>=0.14 (from tensorlayer)
Downloading https://files.pythonhosted.org/packages/34/79/cefff573a53ca3fb4c390739d19541b95f371e24d2990aed4cd8837971f0/scikit_image-0.14.0-cp36-cp36m-manylinux1_x86_64.whl (25.3MB)
安裝sklearn包
[email protected]:~/PycharmProjects/guo/MARTA-GAN-master$ pip install -U scikit-learn
Collecting scikit-learn
Downloading https://files.pythonhosted.org/packages/f9/c8/8db4108aba5e2166cd2ea4eafa1a4b82f89240a1fa85733029cc2358ad1f/scikit_learn-0.19.2-cp36-cp36m-manylinux1_x86_64.whl (4.9MB)
100% |████████████████████████████████| 4.9MB 335kB/s
tensorlayer 1.10.1 has requirement numpy<1.16,>=1.14, but you'll have numpy 1.13.3 which is incompatible.
Installing collected packages: scikit-learn
Found existing installation: scikit-learn 0.19.1
Uninstalling scikit-learn-0.19.1:
Successfully uninstalled scikit-learn-0.19.1
Successfully installed scikit-learn-0.19.2
[email protected]:~/PycharmProjects/guo/MARTA-GAN-master$ python extract_feature.py
/home/gis/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 166, in _parse
return self.parser.parse(argument)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 152, in parse
val = self.convert(argument)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 268, in convert
type(argument)))
TypeError: Expect argument to be a string or int, found <class 'float'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "extract_feature.py", line 27, in <module>
flags.DEFINE_integer("train_size", np.inf, "The size of train images [np.inf]")
File "/home/gis/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/flags.py", line 58, in wrapper
return original_function(*args, **kwargs)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_defines.py", line 315, in DEFINE_integer
DEFINE(parser, name, default, help, flag_values, serializer, **args)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_defines.py", line 81, in DEFINE
DEFINE_flag(_flag.Flag(parser, serializer, name, default, help, **args),
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 107, in __init__
self._set_default(default)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 196, in _set_default
self.default = self._parse(value)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 169, in _parse
'flag --%s=%s: %s' % (self.name, argument, e))
absl.flags._exceptions.IllegalFlagValueError: flag --train_size=inf: Expect argument to be a string or int, found <class 'float'>
[email protected]:~/PycharmProjects/guo/MARTA-GAN-master$ pip install numpy==1.13.3
Collecting numpy==1.13.3
Downloading https://files.pythonhosted.org/packages/57/a7/e3e6bd9d595125e1abbe162e323fd2d06f6f6683185294b79cd2cdb190d5/numpy-1.13.3-cp36-cp36m-manylinux1_x86_64.whl (17.0MB)
100% |████████████████████████████████| 17.0MB 81kB/s
tensorlayer 1.10.1 has requirement numpy<1.16,>=1.14, but you'll have numpy 1.13.3 which is incompatible.
Installing collected packages: numpy
Found existing installation: numpy 1.14.3
Uninstalling numpy-1.14.3:
Successfully uninstalled numpy-1.14.3
Successfully installed numpy-1.13.3
[email protected]:~/PycharmProjects/guo/MARTA-GAN-master$ python train_marta_gan.py
Traceback (most recent call last):
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 166, in _parse
return self.parser.parse(argument)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 152, in parse
val = self.convert(argument)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_argument_parser.py", line 268, in convert
type(argument)))
TypeError: Expect argument to be a string or int, found <class 'float'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train_marta_gan.py", line 26, in <module>
flags.DEFINE_integer("train_size", np.inf, "The size of train images [np.inf]")
File "/home/gis/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/flags.py", line 58, in wrapper
return original_function(*args, **kwargs)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_defines.py", line 315, in DEFINE_integer
DEFINE(parser, name, default, help, flag_values, serializer, **args)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_defines.py", line 81, in DEFINE
DEFINE_flag(_flag.Flag(parser, serializer, name, default, help, **args),
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 107, in __init__
self._set_default(default)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 196, in _set_default
self.default = self._parse(value)
File "/home/gis/anaconda3/lib/python3.6/site-packages/absl/flags/_flag.py", line 169, in _parse
'flag --%s=%s: %s' % (self.name, argument, e))
absl.flags._exceptions.IllegalFlagValueError: flag --train_size=inf: Expect argument to be a string or int, found <class 'float'>
1.錯誤：
D:\Program Files\python3.6\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters
2.原因：（參考：https://wuwb.me/2018/03/h5py-futurewarning-problem-fix.html）
h5py 和 numpy 版本衝突，h5py 官方已修復合併到 master 分支，但是還沒發新版，在發版之前可以用降級 numpy 的方法跳過這個問題。降級命令如下：
pip install numpy==1.13.3(tensorflow-gpu 1.6.0 匹配numpy 1.13.3）
/home/gis/anaconda3/bin/python /home/gis/PycharmProjects/guo/rsisc/resnet50_Transfer_Learning.py
Using TensorFlow backend.
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/rsisc/resnet50_Transfer_Learning.py", line 5, in <module>
from resnet50 import ResNet50
File "/home/gis/PycharmProjects/guo/rsisc/resnet50.py", line 31, in <module>
from keras.applications.imagenet_utils import _obtain_input_shape
ImportError: cannot import name '_obtain_input_shape'
down vote
This issue occured because of the version of keras.
In my case, I was downgrade keras 2.2.2 to 2.2.0, and the problem was solved.
shareimprove this answer
answered Aug 1 at 8:56
user5803658
6113
how do you upgraded kerns 2.2.2 to version less than 2.2.2 – W. Sam Aug 13 at 1:14
Use 'pip install keras==2.2.0' – Raein Hashemi Aug 15 at 17:02
1
@W.Sam you don't have to downgrade Keras, see my answer below. – Geeocode Aug 30 at 16:11
_obtain_input_shape() got an unexpected keyword argument 'include_top'
keras.__version__ == 2.0.9 中，函式 _obtain_input_shape() 的形式：
def _obtain_input_shape(input_shape,
default_size,
min_size,
data_format,
require_flatten,
weights=None):
2）deep-learning-models 案例中，呼叫 _obtain_input_shape() 函式的方式如下：
# Determine proper input shape
input_shape = _obtain_input_shape(input_shape,
default_size=299,
min_size=71,
data_format=K.image_data_format(),
include_top=include_top)
顯然，本文使用的環境中，函式 _obtain_input_shape() 的形參中，沒有關鍵字 include_top 而是 require_flatten。而案例中的程式碼使用的是關鍵字 include_top，這顯然是不可取的。綜上，只要把案例中的 inlcude_top 關鍵字換成 require_flatten 即可。如下所示：
【keras-DeepLearning_Models】_obtain_input_shape() got an unexpected keyword argument 'include_top'
2017年11月11日 16:34:04 Houchaoqun_XMU 閱讀數：3199 標籤： _obtain_input_shape keras Ubuntu inception 更多
個人分類： 【深度學習】
前言：
最近想跑一些主流的網路感受感受。從github上找到了 deep-learning-models 提供的幾個模型，包括：inception-v2, inception-v3, resnet50, vgg16, vgg19 等等。這些程式碼都是基於 keras 框架，正好我最近有在學 tensorflow 和 keras，所以很想跑跑這些程式碼。
心動不如行動，萬事俱備，只欠把程式碼跑起來。此時，出現了一些常見的問題，也正好藉此機會整理下來。問題如下：
1）_obtain_input_shape() got an unexpected keyword argument 'include_top'
2）Exception: URL fetch failure on https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5: None -- [Errno 110] Connection timed out
本文主要分析和整理了第一個問題的解決方案。
第二個問題就是：在下載模型引數檔案的過程中，可能url對應的地址被牆了，導致下載不了。解決辦法就是另想辦法把 resnet50_weights_tf_dim_ordering_tf_kernels.h5 這個檔案下載下來。如果有需要的話，可在本文留言。
Reference：
1）github 原始碼：https://github.com/fchollet/deep-learning-models
2）模型引數資源：https://github.com/fchollet/deep-learning-models/releases
3）相關部落格：http://blog.csdn.net/sinat_26917383/article/details/72982230
4）本文使用的測試資料如下所示：
elephant.jpg
本文使用的 tensorflow 和 keras 的版本：
- tensorflow：
>>> import tensorflow as tf
>>> tf.__version__
'1.1.0'
- keras：
import keras
>>> print keras.__version__
2.0.9
本文實踐步驟如下，以 "resnet50.py" 為例：
1）下載 github 原始碼，原始碼中使用一張名為“elephant.jpg”的影象作為測試。
2）下載測試資料集，上文有提供連結。
3）在原始碼的目錄下執行如下命令：
python resnet50.py
程式報錯，如下所示：
Traceback (most recent call last):
File "resnet50.py", line 289, in <module>
model = ResNet50(include_top=True, weights='imagenet')
File "resnet50.py", line 193, in ResNet50
include_top=include_top)
TypeError: _obtain_input_shape() got an unexpected keyword argument 'include_top'
導致程式報錯的原因分析：
1）keras.__version__ == 2.0.9 中，函式 _obtain_input_shape() 的形式：
def _obtain_input_shape(input_shape,
default_size,
min_size,
data_format,
require_flatten,
weights=None):
2）deep-learning-models 案例中，呼叫 _obtain_input_shape() 函式的方式如下：
# Determine proper input shape
input_shape = _obtain_input_shape(input_shape,
default_size=299,
min_size=71,
data_format=K.image_data_format(),
include_top=include_top)
顯然，本文使用的環境中，函式 _obtain_input_shape() 的形參中，沒有關鍵字 include_top 而是 require_flatten。而案例中的程式碼使用的是關鍵字 include_top，這顯然是不可取的。綜上，只要把案例中的 inlcude_top 關鍵字換成 require_flatten 即可。如下所示：
# Determine proper input shape
input_shape = _obtain_input_shape(input_shape,
default_size=224,
min_size=197,
data_format=K.image_data_format(),
require_flatten=include_top)
再次執行命令，就可以成功執行案例程式碼，如下圖所示：
本文的第二個問題：無法正常下載模型引數
模型引數“resnet50_weights_tf_dim_ordering_tf_kernels.h5”下載地址如下：
連結：http://pan.baidu.com/s/1dE1Lh5J 密碼：puke
需修改的程式碼如下：
# WEIGHTS_PATH = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels.h5'
# load weights
if weights == 'imagenet':
# if include_top:
# weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels.h5',
# WEIGHTS_PATH,
# cache_subdir='models',
# md5_hash='a7b3fe01876f51b976af0dea6bc144eb')
# else:
# weights_path = get_file('resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5',
# WEIGHTS_PATH_NO_TOP,
# cache_subdir='models',
# md5_hash='a268eb855778b3df3c7506639542a6af')
weights_path = './resnet50_weights_tf_dim_ordering_tf_kernels.h5'
model.load_weights(weights_path)
執行結果如下所示：
Predicted: [[(u'n01871265', u'tusker', 0.65325415), (u'n02504458', u'African_elephant', 0.29492217), (u'n02504013', u'Indian_elephant', 0.048155606), (u'n02422106', u'hartebeest', 0.001847562), (u'n02397096', u'warthog', 0.00034257883)]]
由結果可知：p(tusker) = 0.65, p(African_elephant) = 0.29, p(Indian_elephant) = 0.048 ... ... 其中 tusker 的概率是最高的，所以識別結果為 tusker（有長牙的動物，如：象，野豬等）