好記性不如爛筆頭，一直以來都只有寫書面學習筆記的習慣，從來沒寫過部落格。如今很榮幸地加入了浙江大學學生人工智慧協會，立志在跟隨優秀的老師和學長學姐學習AI領域相關技術的同時也為協會的運營和發展貢獻力量。9月份入學以來，因為科研需要加上個人濃烈的興趣，一直堅持著學習機器學習、深度學習相關的知識。如今，我還負責分管協會深度學習論文歸檔這塊的任務，作為協會的資源方便成員的查閱學習。書面筆記不方便資源共享，於是開始寫起了部落格，剛開始嘗試，如部落格有不恰當之處還望海涵。希望此部落格能夠給深度學習有興趣的人一些論文選讀上的參考，少走彎路。此部落格將伴隨著我的學習歷程不定時更新，在如今這個深度學習研究成果爆發產出的時代裡，深度學習論文的發表又多又雜，如有錯誤請及時聯絡我，當然如果有更好的論文推薦，也請告知，不勝感激。

萬事開頭難，本部落格最初的論文，主要從他人的CSDN、部落格園、GitHub等個人部落格或主頁中整理出來。目前的內容主要來自我們協會會長羅浩學長的部落格，在此表示感謝。相關引用的連結我會在文末給出。如下為我讀過論文，我會盡量對我讀過每篇優秀論文寫閱讀筆記（整理中），若有錯誤之處，還望指正。

深度學習的基礎

• Hecht-Nielsen R. Theory of the backpropagation neural network[J]. Neural Networks, 1988, 1(Supplement-1): 445-448.（BP神經網路）[PDF]
• Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets.[J]. Neural Computation, 2006, 18(7): 1527-1554.（深度學習的開端DBN）
  [PDF]
• Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks.[J]. Science, 2006, 313(5786): 504-7.（自編碼器降維）[PDF]
• Ng A. Sparse autoencoder[J]. CS294A Lecture notes, 2011, 72(2011): 1-19.（稀疏自編碼器）[PDF]
• Vincent P, Larochelle H, Lajoie I, et al. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion[J]. Journal of Machine Learning Research, 2010, 11(Dec): 3371-3408.（堆疊自編碼器，SAE）
  [PDF]

深度學習爆發：從AlexNet到Capsules

• Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems. 2012.（AlexNet）[PDF]
• Simonyan, Karen, and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409.1556 (2014).（VGGNet）[PDF]
• Szegedy, Christian, et al. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. （GoogLeNet）[PDF]
• Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the Inception Architecture for Computer Vision[J]. Computer Science, 2015: 2818-2826.（Inception-V3）[PDF]
• He, Kaiming, et al. Deep residual learning for image recognition. arXiv preprint arXiv: 1512.03385 (2015).（ResNet）[PDF]
• Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions[J]. arXiv preprint arXiv: 1610.02357, 2016.（Xception）[PDF]
• Huang G, Liu Z, Weinberger K Q, et al. Densely Connected Convolutional Networks[J]. 2016.  (DenseNet)[PDF]
• Squeeze-and-Excitation Networks. （SeNet）[PDF]
• Zhang X, Zhou X, Lin M, et al. Shufflenet: An extremely efficient convolutional neural network for mobile devices[J]. arXiv preprint arXiv: 1707.01083, 2017.（Shufflenet）[PDF]
• Sabour S, Frosst N, Hinton G E. Dynamic routing between capsules[C].Advances in Neural Information Processing Systems. 2017: 3859-3869.（Capsules）[PDF]

深度學習中非常有用的Tricks

• Srivastava N, Hinton G E, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.（Dropout）[PDF]
• Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[J]. arXiv preprint arXiv: 1502.03167, 2015.（Batch Normalization）[PDF]
• Lin M, Chen Q, Yan S. Network In Network[J]. Computer Science, 2014.（Global average pooling）[PDF]

遞迴神經網路RNN

• Mikolov T, Karafiát M, Burget L, et al. Recurrent neural network based language model[C].Interspeech. 2010, 2: 3.（RNN和語language model結合較經典文章）[PDF]
• Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.（LSTM的數學原理）[PDF]
• Chung J, Gulcehre C, Cho K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXiv preprint arXiv: 1412.3555, 2014.（GRU網路）[PDF]

生成對抗網路GAN

• Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C].Advances in neural information processing systems. 2014: 2672-2680.（GAN）[PDF]
• Mirza M, Osindero S. Conditional generative adversarial nets[J]. arXiv preprint arXiv: 1411.1784, 2014.（CGAN）[PDF]
• Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint arXiv: 1511.06434, 2015.（DCGAN）[PDF]
• Denton E L, Chintala S, Fergus R. Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks[C].Advances in neural information processing systems. 2015: 1486-1494.（LAPGAN）[PDF]
• Chen X, Duan Y, Houthooft R, et al. Infogan: Interpretable representation learning by information maximizing generative adversarial nets[C].Advances in Neural Information Processing Systems. 2016: 2172-2180.（InfoGAN）[PDF]
• Arjovsky M, Chintala S, Bottou L. Wasserstein gan[J]. arXiv preprint arXiv: 1701.07875, 2017.（WGAN）[PDF]
• Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[J]. arXiv preprint arXiv: 1703.10593, 2017.（CycleGAN）[PDF]
• Yi Z, Zhang H, Gong P T. DualGAN: Unsupervised Dual Learning for Image-to-Image Translation[J]. arXiv preprint arXiv: 1704.02510, 2017.（DualGAN）[PDF]
• Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks[J]. arXiv preprint arXiv: 1611.07004, 2016.（pix2pix）[PDF]

遷移學習

• Fei-Fei L, Fergus R, Perona P. One-shot learning of object categories[J]. IEEE transactions on pattern analysis and machine intelligence, 2006, 28(4): 594-611.（One shot learning）[PDF]
• Larochelle H, Erhan D, Bengio Y. Zero-data learning of new tasks[J]. 2008: 646-651.（Zero shot learning）[PDF]

目標檢測

• Szegedy C, Toshev A, Erhan D. Deep neural networks for object detection[C].Advances in Neural Information Processing Systems. 2013: 2553-2561.（深度學習早期的物體檢測）[PDF]
• Girshick, Ross, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition. 2014.（R-cnn）[PDF]
• He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[C].European Conference on Computer Vision. Springer International Publishing, 2014: 346-361.（SPPNet）[PDF]
• Girshick R. Fast r-cnn[C]. Proceedings of the IEEE International Conference on Computer Vision. 2015: 1440-1448.（Fast R-cnn）[PDF]
• Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[C]. Advances in neural information processing systems. 2015: 91-99.（Faster R-cnn）[PDF]
• Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 779-788.（YOLO）[PDF]
• Liu W, Anguelov D, Erhan D, et al. SSD: Single shot multibox detector[C].European Conference on Computer Vision. Springer International Publishing, 2016: 21-37.（SSD）[PDF]
• Li Y, He K, Sun J. R-fcn: Object detection via region-based fully convolutional networks[C].Advances in Neural Information Processing Systems. 2016: 379-387.（R-fcn）[PDF]

語義分割

• Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 3431-3440.（最經典的FCN）[PDF]
• Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. arXiv preprint arXiv: 1606.00915, 2016.（DeepLab）[PDF]
• Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[J]. arXiv preprint arXiv: 1612.01105, 2016.（PSPNet）[PDF]
• He K, Gkioxari G, Dollár P, et al. Mask R-CNN[J]. arXiv preprint arXiv: 1703.06870, 2017.（MASK R-cnn）[PDF]
• Hu R, Dollár P, He K, et al. Learning to Segment Every Thing[J]. arXiv preprint arXiv: 1711.10370, 2017.（Mask R-cnn增強版） [PDF]

影象壓縮

• George Toderici, Sean M. O' Malley, Sung Jin Hwang, Damien Vincent, David Minnen, Shumeet Baluja, Michele Covell, and Rahul Sukthankar. Variable rate image compression with recurrent neural networks. In ICLR, 2016.（深度學習運用在影象壓縮上的一篇經典論文，RNN模型）[PDF]
• George Toderici, Damien Vincent, Nick Johnston, Sung Jin Hwang, David Minnen, Joel Shor, and Michele Covell. Full resolution image compression with recurrent neural networks. arXiv preprint  arXiv: 1608.05148, 2016.（提出的RNN網路首次在Kodak資料集上超越JPEG）[PDF]
• Mohammad Haris Baig, Vladlen Koltun, Lorenzo Torresani. Learn to Inpaint for Image Compression. In NIPS, 2017.[PDF]
• Feng Jiang, Wen Tao, Shaohui Liu, Jie Ren, Xun Guo, Debin Zhao. An End-to-End Compression Framework Based on Convolutional Neural Networks.（CNN在影象壓縮中的運用）[PDF]

關鍵點/姿態檢測

• Shih-En Wei, Varun Ramakrishna, Takeo Kanade, Yaser Sheikh. Convolutional Pose Machines. CVPR, 2016.（經典的關鍵點檢測的論文,在2016年MPII姿態分析競賽中位列第二,也是我的第一次參加天池比賽在FashionAI服飾關鍵點定位賽中用到的模型）[PDF]
• Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked Hourglass Networks for Human Pose Estimation.（非常有名,特徵多尺度,速度快,在2016年MPII姿態分析競賽中位列榜首,在FashionAI天池大賽中中也被很多隊伍用到)[PDF]
• W. Wang, Y. Xu, J. Shen, and S.-C. Zhu,Attentive Fashion Grammar Network for Fashion Landmark Detection and Clothing Category Classification.CVPR, 2018.(最新的FashionAI領域的大作,提出兩種位置關係語法,雙向卷積RNN網路資訊傳遞模型,針對不同認為提出的兩種attention機制,思想非常fancy,值得一讀.)[PDF]

ReID

• Ding S, Lin L, Wang G, et al. Deep feature learning with relative distance comparison for person re-identification[J]. Pattern Recognition, 2015, 48(10): 2993-3003.[PDF]（triplet loss）
• Hermans A, Beyer L, Leibe B. In Defense of the Triplet Loss for Person Re-Identification[J]. arXiv preprint arXiv:1703.07737, 2017.[PDF]（Triplet loss with hard mining sample）
• Chen W, Chen X, Zhang J, et al. Beyond triplet loss: a deep quadruplet network for person re-identification[J]. arXiv preprint arXiv:1704.01719, 2017.[PDF]（四元組）
• Qiqi Xiao, Hao Luo, Chi Zhang. Margin Sample Mining Loss: A Deep Learning Based Method for Person Re-identification[J]. arXiv preprint arXiv: 1710.00478.[PDF]（MSML）
• Zhang X, Luo H, Fan X, et al. AlignedReID: Surpassing Human-Level Performance in Person Re-Identification[J]. arXiv preprint arXiv:1711.08184, 2017. [PDF]（AlignedReid，首次超越人類）

引用連結

• http://blog.csdn.net/qq_21190081/article/details/69564634
• http://github.com/michuanhaohao/paper
• http://github.com/RedditSota/state-of-the-art-result-for-machine-learning-problems
• http://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap
• http://github.com/kjw0612/awesome-deep-vision

-------------------------------------------

Youzhi Gu, master student

Foresight Control Center

College of Control Science & Engineering

Zhejiang University

Email: [email protected]