Save and Load Machine Learning Models in Python with scikit
Hello Jason,
I am new to machine learning. I am your big fan and read a lot of your blog and books. Thank you very much for teaching us machine learning.
I tried to pickle my model but fail. My model is using VGG16 and replace the top layer for my classification solution. I further narrowed down the problem and find that it is the VGG16 model failed to pickle. Please find my simplified code below and error log below:
It will be highly appreciated if you can give me some direction on how to fix this error.
Thank you very much
———————————————————-
# Save Model Using Pickle
from keras.applications.vgg16 import VGG16
import pickle
model = VGG16(weights=’imagenet’, include_top=False)
filename = ‘finalized_model.sav’
pickle.dump(model, open(filename, ‘wb’))
—————————————————-
/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 /Users/samueltin/Projects/bitbucket/share-card-ml/pickle_test.py
Using TensorFlow backend.
Traceback (most recent call last):
File “/Users/samueltin/Projects/bitbucket/share-card-ml/pickle_test.py”, line 8, in
pickle.dump(model, open(filename, ‘wb’))
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 1376, in dump
Pickler(file, protocol).dump(obj)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 224, in dump
self.save(obj)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 331, in save
self.save_reduce(obj=obj, *rv)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 425, in save_reduce
save(state)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 606, in save_list
self._batch_appends(iter(obj))
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 621, in _batch_appends
save(x)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 331, in save
self.save_reduce(obj=obj, *rv)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 425, in save_reduce
save(state)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 606, in save_list
self._batch_appends(iter(obj))
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 621, in _batch_appends
save(x)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 331, in save
self.save_reduce(obj=obj, *rv)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 425, in save_reduce
save(state)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 331, in save
self.save_reduce(obj=obj, *rv)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 425, in save_reduce
save(state)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 331, in save
self.save_reduce(obj=obj, *rv)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 425, in save_reduce
save(state)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 606, in save_list
self._batch_appends(iter(obj))
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 621, in _batch_appends
save(x)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 331, in save
self.save_reduce(obj=obj, *rv)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 425, in save_reduce
save(state)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 331, in save
self.save_reduce(obj=obj, *rv)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 425, in save_reduce
save(state)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 331, in save
self.save_reduce(obj=obj, *rv)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 425, in save_reduce
save(state)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 606, in save_list
self._batch_appends(iter(obj))
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 621, in _batch_appends
save(x)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 568, in save_tuple
save(element)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 286, in save
f(self, obj) # Call unbound method with explicit self
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 655, in save_dict
self._batch_setitems(obj.iteritems())
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 669, in _batch_setitems
save(v)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py”, line 306, in save
rv = reduce(self.proto)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy_reg.py”, line 70, in _reduce_ex
raise TypeError, “can’t pickle %s objects” % base.__name__
TypeError: can’t pickle module objects
Process finished with exit code 1
Reply相關推薦
Save and Load Machine Learning Models in Python with scikit
Hello Jason, I am new to machine learning. I am your big fan and read a lot of your blog and books. Thank you very much for teaching us machine le
Training Machine Learning Models in Pharma and Biotech Manufacturing with Bigfinite Amazon Web Services
Creating and training machine learning models has become less time consuming and more cost efficient thanks to technology advancements like open source sof
Learn How to Code and Deploy Machine Learning Models on Spark Structured Streaming
This post is a token of appreciation for the amazing open source community of Data Science, to which I owe a lot of what I have learned. For last few month
Top 5 Machine Learning Libraries in Python
(Sponsors) Get started learning Python with DataCamp's free Intro to Python tutorial. Learn Data Science by completing interactive coding challenges and
How to rapidly test dozens of deep learning models in Python
Although k-fold cross validation is a great way of assessing a model’s performance, it’s computationally expensive to obtain these results. We can simply s
Rescaling Data for Machine Learning in Python with Scikit
Tweet Share Share Google Plus Your data must be prepared before you can build models. The data p
How to Load Data in Python with Scikit
Tweet Share Share Google Plus Before you can build machine learning models, you need to load you
Feature Selection in Python with Scikit
Tweet Share Share Google Plus Not all data attributes are created equal. More is not always bett
Save And Finalize Your Machine Learning Model in R
Tweet Share Share Google Plus Finding an accurate machine learning is not the end of the project
How To Load CSV Machine Learning Data in Weka (如何在Weka中載入CSV機器學習資料)
How To Load CSV Machine Learning Data in Weka 原文作者：Jason Brownlee 原文地址：https://machinelearningmastery.com/load-csv-machine-learning-data-weka/
How Facebook Uses Bayesian Optimization to Conduct Better Experiments in Machine Learning Models
How Facebook Uses Bayesian Optimization to Conduct Better Experiments in Machine Learning ModelsHyperparameter optimization is a key aspect of the lifecycl
Applitools Recognized as a Top Artificial Intelligence and Machine Learning Solution in DevOps
According to the report, AI is now the number one strategic enterprise IT investment priority in 2018. Applitools developed the first and only AI-powered i
Understanding Data and Machine Learning Models with Visualizations (Part 1)
Examining Feature-PC CorrelationOn the non-interactive side, the tool also generates heatmaps with additional information about the principal components. T
Train TF models in Python and Invoke models in Java
Plan A #Train in Python import tensorflow as tf # good idea # https://stackoverflow.com/documentation/tensorflow/10718/save-te
How to Normalize and Standardize Your Machine Learning Data in Weka
Tweet Share Share Google Plus Machine learning algorithms make assumptions about the dataset you
How to Build an Ensemble Of Machine Learning Algorithms in R (ready to use boosting, bagging and stacking)
Tweet Share Share Google Plus Ensembles can give you a boost in accuracy on your dataset. In thi
Prepare Data for Machine Learning in Python with Pandas
Tweet Share Share Google Plus If you are using the Python stack for studying and applying machin
How to Load and Explore Time Series Data in Python
Tweet Share Share Google Plus The Pandas library in Python provides excellent, built-in support
tensorflow data's save and load
int numpy spa pat clas temp 天下 col you note: if you‘ll load data,the data shape should be similar with saved data‘s shape. -- 中式英語，天下無