Essential Resources for Deep Learning

(Stay tuned, as I keep updating the post while I grow and plow in my deep learning garden:))

(See this page for a machine learning glossary. See this page for deep learning glossary.)

(Google TensorFlow？Caffe？Amazon MXNet？Keras？PyTorch？Microsoft CNTK?  Check out this page for the comparisons among the deep learning libraries that will guide you in picking the most appropriate framework(s) for your own project(s).)

The Coursera videos and Michael Nielsen’s book have helped me a lot to get the fundamentals right.

If you seek a more advanced book after finishing the course and book I mentioned above, Deep Learning (MIT Press ) by Ian Goodfellow and Yoshua Bengio and Aaron Courville might be a good choice.

If you want to start implementing deep networks, you will need to learn tools like Google TensorFlow or Keras or Microsoft CNTK or Caffe or Theano. I would recommend TensorFlow (which is pretty new and is constantly improving with time); Also, Keras has TensorFlow as backend (the other backend of Keras is Theano), this can be a good choice for fast experiments of your deep learning ideas.

If you want to do deep learning on clusters, you will probably need to learn Apache Spark or Hadoop. Getting a good handle on these tools is probably as important as learning about the math behind the models.

I recommend you look at Dr. Andy Thomas’s great tutorial: Python TensorFlow Tutorial – Build a Neural Network (pdf). You will get a pretty good sense of how TensorFlow works. If you have played with machine learning and TensorFlow for a while, I recommend reading his tips and tricks post (pdf).  See Andy’s Neural Networks Tutorial – A Pathway to Deep Learning (pdf) for the introduction to some essential concepts involved in deep learning. The good thing is that for most concepts it provides Python code snippets. See his another post Convolutional Neural Networks Tutorial in TensorFlow (pdf) for a very good illustration and explanation about how CNN works and it is implementation in TensorFlow. In the post Keras tutorial – build a convolutional neural network in 11 lines (pdf) (May 17, 2017), Andy provides a pretty good explanation of how to build a CNN model using Keras along with code snippet, and the full code of this Keras tutorial can be found here. (Check here for his GitHub repo for all the code involved on his blog.)

Check out this GIF animation on Justin Francis’ twitter for a visual and intuitive training evolution of deep neural networks. If you want to dig into more details about this, check out his tutorial: Not another MNIST tutorial with TensorFlow (pdf).

I also recommend to check out this recent nice post 41 Essential Machine Learning Interview Questions (with answers) (pdf)(Jan 9, 2017) by Roger Huang.

Check out the two great posts about deep learning written by Keras father Francois Chollet:

Apart from the popular TensorFlow and Keras, Microsoft’s CNTK is another good option for deep learning. Check out this post (August 3, 2017) by Andy Thomas for a tutorial (in Python) for Microsoft CNTK to build a neural network (pdf).

 

======Below are some good courses and resources about deep learning (and TensorFlow):

Watch the videos in this way

2016 lectures 1-8

2015 lecture 8 (equivalent to missing 2016 lecture 9)

2016 lectures 10-11

2015 lecture 13, 16, guest lectures

2016 lecture slides for May 17, 19, 24, 26

This tutorial introduced basic concepts involved in deep learning (i.e., deep neural nets, MLP, RNN, LSTM).

This is a pretty good introduction to TensorFlow usage.

 

An 3-hour course (video + slides) offers developers a quick introduction to deep learning fundamentals, with TensorFlow.

Check this code and this code out for an example of how to display a graphical visualization and you should also see training data in the terminal.

 

======Below provides some great videos about TensorFlow and deep learning

With TensorFlow, deep machine learning transitions from an area of research to mainstream software engineering. In this video, Martin Gorner demonstrates how to construct and train a neural network that recognises handwritten digits. Along the way, he’ll describe some “tricks of the trade” used in neural network design, and finally, he’ll bring the recognition accuracy of his model above 99%.

Content applies to software developers of all levels. Experienced machine learning enthusiasts, this video will introduce you to TensorFlow through well known models such as dense and convolutional networks. This is an intense technical video designed to help beginners in machine learning ramp up quickly.

Missed the conference? Watch all the talks here: https://goo.gl/c1Vs3h
Watch more talks about Big Data & Machine Learning here: https://goo.gl/OcqI9k

You can now run this yourself with a self-paced code lab: https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0

 

Deep learning has already revolutionized machine learning research, but it hasn’t been broadly accessible to many developers. In this video, Martin Gorner explores the possibilities of recurrent neural networks by building a language model in TensorFlow. What this model can do will impress you.

Developers with no prior machine learning experience are welcome. We do recommend that you watch the video “”TensorFlow and deep learning without a PhD part 1″” unless you already know about dense and convolutional networks and are only interested in recurrent networks.

This video is an intense technical video designed to help beginners in machine learning ramp up quickly.

Here is the Link to his github code https://github.com/martin-gorner/tensorflow-mnist-tutorial

Missed the conference? Watch all the talks here: https://goo.gl/c1Vs3h
Watch more talks about Big Data & Machine Learning here: https://goo.gl/OcqI9k

In this video, Yufeng Guo applies deep learning models to local prediction on mobile devices. Yufeng shows you how to use TensorFlow to implement a machine learning model that is tailored to a custom dataset. You will come away knowing enough to get started solving your own deep learning problems.

Missed the conference? Watch all the talks here: https://goo.gl/c1Vs3h
Watch more talks about Big Data & Machine Learning here: https://goo.gl/OcqI9k

 

======Deep learning papers:

  • The most cited deep learning papers (GitHub repo link. Thanks for Davide‘s sharing.)

======See below for more resources about deep learning