Suggested Readings for Learning "Deep"

Abstract

This document lists some on-line resources for the readers who are interested in learning machine learning and deep learning.
As per my experience, taking a well arranged on-line course, i.e., this one, would be the most efficient way to solidly learn machine learning.
In addition, here also lists some advance materials for well understanding deep learning in terms of practical and theoretical aspects.

The Basics

  • Linear algebra: https://www.khanacademy.org/math/linear-algebra
    This is a prerequisite for machine learning.

  • Machine learning: https://www.coursera.org/learn/machine-learning/
    If you decide having a solid knowledge of ML, taking this course will be an efficient way to do so. During this course, the assigned matlab homeworks are suggested to be done for a better understanding.
    Another ML website: https://goo.gl/itg1y8

    Machine Learning:http://www.holehouse.org/mlclass/index.html

  • Deep learning with less material: http://ufldl.stanford.edu/tutorial/
    If you already have some basic knowledge of machine learning and desire moving to deep learning earlier, one of a great choices is to follow this tutorial which focuses on only the prerequisite of deep learning. In addition, I also suggest you to finish the related matlab assignments.

  • Deep learning with rich material: http://deeplearning.net/tutorial/contents.html
    This website also teaches the deep learning but with a different programing language python for implementation. This place also contains rich details of deep learning structure and relevant theories.

Others

  • Stanford course - CS224d Deep Learning for Natural Language Processing:
    http://cs224d.stanford.edu/
  • Stanford course - CS229 Machine Learning:
    http://cs229.stanford.edu/
  • Stanford course - CS231n Convolutional Neural Networks for Visual Recognition:
    http://cs231n.stanford.edu/
  • VGG model application:
    http://www.robots.ox.ac.uk/~vgg/practicals/cnn/index.html
  • CVPR2015 application:
    http://www.pamitc.org/cvpr15/program.php/
  • deep learning introduction(chinese):
    http://blog.csdn.net/zouxy09/article/details/8781543/

Suggested Reading List

Caffe

  • Caffe tutorial:
    http://vision.princeton.edu/courses/COS598/2015sp/slides/Caffe/caffe_tutorial.pdf
  • Feature extraction:
    http://caffe.berkeleyvision.org/gathered/examples/feature_extraction.html

References

  1. Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor, Rob Fergus. Deconvolutional networks. In In CVPR. (2010).

  2. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580 [cs] (2012). Link

  3. Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. 1097–1105 In Advances in Neural Information Processing Systems 25. Curran Associates, Inc., 2012. Link

  4. Matthew D. Zeiler, Rob Fergus. Visualizing and Understanding Convolutional Networks. arXiv:1311.2901 [cs] (2013). Link

  5. Li Deng. A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Transactions on Signal and Information Processing 3 (2014). Link

  6. Karen Simonyan, Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv:1409.1556 [cs] (2014). Link