(Stay tuned, as I keep updating the post while I grow and plow in my deep learning garden:))
(See this page for a machine learning glossary. See this page for deep learning glossary.)
(Google TensorFlow?Caffe?Amazon MXNet?Keras?PyTorch?Microsoft CNTK? Check out this page for the comparisons among the deep learning libraries that will guide you in picking the most appropriate framework(s) for your own project(s).)
The Coursera videos and Michael Nielsen’s book have helped me a lot to get the fundamentals right.
- Andrew Ng’s Coursera course is a great start for the fundamentals – Machine Learning | Coursera
- Michael Nielsen’s book Neural networks and deep learning is a good addendum for the Coursera videos. It is a nice introductory book on deep learning which I strongly recommend.
If you seek a more advanced book after finishing the course and book I mentioned above, Deep Learning (MIT Press ) by Ian Goodfellow and Yoshua Bengio and Aaron Courville might be a good choice.
If you want to start implementing deep networks, you will need to learn tools like Google TensorFlow or Keras or Microsoft CNTK or Caffe or Theano. I would recommend TensorFlow (which is pretty new and is constantly improving with time); Also, Keras has TensorFlow as backend (the other backend of Keras is Theano), this can be a good choice for fast experiments of your deep learning ideas.
If you want to do deep learning on clusters, you will probably need to learn Apache Spark or Hadoop. Getting a good handle on these tools is probably as important as learning about the math behind the models.
I recommend you look at Dr. Andy Thomas’s great tutorial: Python TensorFlow Tutorial – Build a Neural Network (pdf). You will get a pretty good sense of how TensorFlow works. If you have played with machine learning and TensorFlow for a while, I recommend reading his tips and tricks post (pdf). See Andy’s Neural Networks Tutorial – A Pathway to Deep Learning (pdf) for the introduction to some essential concepts involved in deep learning. The good thing is that for most concepts it provides Python code snippets. See his another post Convolutional Neural Networks Tutorial in TensorFlow (pdf) for a very good illustration and explanation about how CNN works and it is implementation in TensorFlow. In the post Keras tutorial – build a convolutional neural network in 11 lines (pdf) (May 17, 2017), Andy provides a pretty good explanation of how to build a CNN model using Keras along with code snippet, and the full code of this Keras tutorial can be found here. (Check here for his GitHub repo for all the code involved on his blog.)
Check out this GIF animation on Justin Francis’ twitter for a visual and intuitive training evolution of deep neural networks. If you want to dig into more details about this, check out his tutorial: Not another MNIST tutorial with TensorFlow (pdf).
I also recommend to check out this recent nice post 41 Essential Machine Learning Interview Questions (with answers) (pdf)(Jan 9, 2017) by Roger Huang.
Check out the two great posts about deep learning written by Keras father Francois Chollet:
- Part 1: The limitations of deep learning (July 17, 2017) — pdf
- Part 2: The future of deep learning (July 18, 2017) — pdf
Apart from the popular TensorFlow and Keras, Microsoft’s CNTK is another good option for deep learning. Check out this post (August 3, 2017) by Andy Thomas for a tutorial (in Python) for Microsoft CNTK to build a neural network (pdf).
======Below are some good courses and resources about deep learning (and TensorFlow):
Watch the videos in this way
2016 lectures 1-8
2015 lecture 8 (equivalent to missing 2016 lecture 9)
2016 lectures 10-11
2015 lecture 13, 16, guest lectures
2016 lecture slides for May 17, 19, 24, 26
This tutorial introduced basic concepts involved in deep learning (i.e., deep neural nets, MLP, RNN, LSTM).
- Book: Machine Learning with TensorFlow (Github, tensorflowbook)
- Book: TensorFlow for Machine Intelligence (Github repo, book webpage)
- Book: Fundamentals of Deep Learning (Github repo)
- A course offered at Stanford University: TensorFlow for Deep Learning Research (Course syllabus, Github repo)
This is a pretty good introduction to TensorFlow usage.
- Lecture note 2: TensorFlow Ops (pdf, it introduced to TensorBoard usage)
An 3-hour course (video + slides) offers developers a quick introduction to deep learning fundamentals, with TensorFlow.
Check this code and this code out for an example of how to display a graphical visualization and you should also see training data in the terminal.
- Introduction to TensorFlow (pdf) – This is a pretty good and detailed instruction of TensorFlow usage. See more on Deep Learning Tutorials (e.g., Super-resolution (Jiwon’s slide) –pdf)
- Tensorflow – Intro (Mar2, 2017) – Slides and Github code (Inputs_types.ipynb).
======Below provides some great videos about TensorFlow and deep learning
- TensorFlow and Deep Learning without a PhD, Part 1 (Google Cloud Next ’17) by Martin Görner (Published on Mar 8, 2017) – Great explanation. (You can now run this yourself with a self-paced code lab).
With TensorFlow, deep machine learning transitions from an area of research to mainstream software engineering. In this video, Martin Gorner demonstrates how to construct and train a neural network that recognises handwritten digits. Along the way, he’ll describe some “tricks of the trade” used in neural network design, and finally, he’ll bring the recognition accuracy of his model above 99%.
Content applies to software developers of all levels. Experienced machine learning enthusiasts, this video will introduce you to TensorFlow through well known models such as dense and convolutional networks. This is an intense technical video designed to help beginners in machine learning ramp up quickly.
Missed the conference? Watch all the talks here: https://goo.gl/c1Vs3h
Watch more talks about Big Data & Machine Learning here: https://goo.gl/OcqI9k
You can now run this yourself with a self-paced code lab: https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0
- TensorFlow and Deep Learning without a PhD, Part 2 (Google Cloud Next ’17) by Martin Görner (Published on Mar 10, 2017) – very informative (Link to his GitHub code).
Deep learning has already revolutionized machine learning research, but it hasn’t been broadly accessible to many developers. In this video, Martin Gorner explores the possibilities of recurrent neural networks by building a language model in TensorFlow. What this model can do will impress you.
Developers with no prior machine learning experience are welcome. We do recommend that you watch the video “”TensorFlow and deep learning without a PhD part 1″” unless you already know about dense and convolutional networks and are only interested in recurrent networks.
This video is an intense technical video designed to help beginners in machine learning ramp up quickly.
Here is the Link to his github code https://github.com/martin-gorner/tensorflow-mnist-tutorial
Missed the conference? Watch all the talks here: https://goo.gl/c1Vs3h
Watch more talks about Big Data & Machine Learning here: https://goo.gl/OcqI9k
- On-device machine learning: TensorFlow on Android (Google Cloud Next ’17) by Yufeng Guo (Published on Mar 10, 2017) – The explanation of CNN in this talk is pretty good.
In this video, Yufeng Guo applies deep learning models to local prediction on mobile devices. Yufeng shows you how to use TensorFlow to implement a machine learning model that is tailored to a custom dataset. You will come away knowing enough to get started solving your own deep learning problems.
Missed the conference? Watch all the talks here: https://goo.gl/c1Vs3h
Watch more talks about Big Data & Machine Learning here: https://goo.gl/OcqI9k
======Deep learning papers:
- The most cited deep learning papers (GitHub repo link. Thanks for Davide‘s sharing.)
======See below for more resources about deep learning
- Deep learning weekly (digest)
- Machine Learning is Fun! Part 3: Deep Learning and Convolutional Neural Networks
- Brandon Rohrer
- Awesome deep learning – A curated list of awesome Deep Learning tutorials, projects and communities.
- Datasets
- Researchers
- WebSites
- Frameworks
- Free Online Books
- Courses
- Videos and Lectures
- Papers
- Tutorials
- Miscellaneous
- Interview Resources
- Artificial Intelligence
- Genetic Algorithms
- Statistics
- Useful Blogs
- Resources on Quora
- Resources on Kaggle
- Cheat Sheets
- Classification
- Linear Regression
- Logistic Regression
- Model Validation using Resampling
- Deep Learning
- Natural Language Processing
- Computer Vision
- Support Vector Machine
- Reinforcement Learning
- Decision Trees
- Random Forest / Bagging
- Boosting
- Ensembles
- Stacking Models
- VC Dimension
- Bayesian Machine Learning
- Semi Supervised Learning
- Optimizations
- Other Useful Tutorials