Keras Official docs
Keras is a Python library for deep learning that runs on TensorFlow, Theano or CNTK. http://keras.io/
Keras is the ideal library for rapid experimentation. Currently, the biggest downside to Keras is that it doesn’t support multi-GPU environments for parallel training (see this discussion).
Keras was developed and maintained by François Chollet, a Google engineer using four guiding principles:
- Modularity: A model can be understood as a sequence or a graph alone. All the concerns of a deep learning model are discrete components that can be combined in arbitrary ways.
- Minimalism: The library provides just enough to achieve an outcome, no frills and maximizing readability.
- Extensibility: New components are intentionally easy to add and use within the framework, intended for researchers to trial and explore new ideas.
- Python: No separate model files with custom file formats. Everything is native Python.
Keras examples – General & Basics
- Develop Your First Neural Network in Python With Keras Step-By-Step (By Jason Brownlee on May 24, 2016)
In this post, you discovered how to create your first neural network model using the powerful Keras Python library for deep learning.
Specifically, you learned the five key steps in using Keras to create a neural network or deep learning model, step-by-step including:
- How to load data.
- How to define neural network in Keras.
- How to compile a Keras model using the efficient numerical backend.
- How to train a model on data.
- How to evaluate a model on data.
- Installing Keras with TensorFlow backend
- 9 Ways to Get Help with Deep Learning in Keras (by Jason Brownlee on August 2, 2017 in Deep Learning)
-
Keras Tutorial: The Ultimate Beginner’s Guide to Deep Learning in Python
This tutorial is pretty good. it provides some good links to some basic concepts of machine learning.
- Introduction to Python Deep Learning with Keras (by on May 10, 2016)
- 5 Step Life-Cycle for Neural Network Models in Keras (by Jason Brownlee on August 11, 2016 in Deep Learning)
- Regression Tutorial with the Keras Deep Learning Library in Python (by on June 9, 2016 in Deep Learning)
- How to Grid Search Hyperparameters for Deep Learning Models in Python With Keras (by on August 9, 2016 in Deep Learning)
Tips for Hyperparameter Optimization
Below are some handy tips to consider when tuning hyperparameters of your neural network.
- k-fold Cross Validation. You can see that the results from the examples in this post show some variance. A default cross-validation of 3 was used, but perhaps k=5 or k=10 would be more stable. Carefully choose your cross validation configuration to ensure your results are stable.
- Review the Whole Grid. Do not just focus on the best result, review the whole grid of results and look for trends to support configuration decisions.
- Parallelize. Use all your cores if you can, neural networks are slow to train and we often want to try a lot of different parameters. Consider spinning up a lot of AWS instances.
- Use a Sample of Your Dataset. Because networks are slow to train, try training them on a smaller sample of your training dataset, just to get an idea of general directions of parameters rather than optimal configurations.
- Start with Coarse Grids. Start with coarse-grained grids and zoom into finer grained grids once you can narrow the scope.
- Do not Transfer Results. Results are generally problem specific. Try to avoid favorite configurations on each new problem that you see. It is unlikely that optimal results you discover on one problem will transfer to your next project. Instead look for broader trends like number of layers or relationships between parameters.
- Reproducibility is a Problem. Although we set the seed for the random number generator in NumPy, the results are not 100% reproducible. There is more to reproducibility when grid searching wrapped Keras models than is presented in this post.
- Time Series Prediction With Deep Learning in Keras (by on July 19, 2016 in Deep Learning)
Keras examples – Images
- Multi-label classification with Keras (by on May 7, 2018 in Deep Learning, Keras, Tutorials ) — pdf
- ImageNet classification with Python and Keras By Adrian Rosebrock on August 10, 2016
- Building powerful image classification models using very little data (pdf)
- Image Recognition in Python with Keras (by Greg | October 13, 2016) (pdf)
- Heat map of CNN output (CNN Localization)
- Implementation of the VGG-CAM model with Keras – GitHub repo (Original Matlab implementation (GitHub repo) and paper here.)
Keras examples – Text mining & NLP
- How to read: Character level deep learning (22 Jun 2016)
After reading a few papers about NLP, and specifically deep learning applications, the author decided to go ahead and try out a few things on my own. In this post the author will demonstrate a character level models for sentiment classification. The models are built with the author’s favourite framework Keras (with Tensorflow as back-end). In case you haven’t used Keras before I strongly suggest it, it is simple and allows for very fast prototyping (thanks François Chollet. After version 1.0 and the introduction of the new functional API, creating complex models can be as easy as a few lines. I’m hoping to demonstrate some of it’s potential as we go along.
If you want to get familiar with the framework the author would suggest following these links:
- 30 seconds to Keras
- Keras examples
- Building powerful image classification models using very little data
Assume that most people reading this have some basic knowledge about convolution networks, mlps, and rnn/lstm models.
A very popular post from Andrej Karpathy talking about the effectiveness of recurrent nets presents a character level language model built with RNNs, find it here. A simple implementation of this model for Keras, here.
- A Keras example for text classification – where they have 8 output classes and use a bag of words (GitHub repo, see here for more Keras examples)
- Predict Sentiment From Movie Reviews Using Deep Learning (by on July 4, 2016 in Deep Learning)
- This example demonstrates the use of Convolution1D for text classification with Keras: https://github.com/fchollet/keras/blob/master/examples/imdb_cnn.py
- Architecture for Text Classification (suggestion on Github)
- Time Series Prediction with LSTM Recurrent Neural Networks in Python with Keras (by Jason Brownlee onJuly 21, 2016)
- Sequence Classification with LSTM Recurrent Neural Networks in Python with Keras (by Jason Brownlee onJuly 26, 2016)
- Multi-Class Classification Tutorial with the Keras Deep Learning Library (by Jason Brownlee on June 2,2016)
In this post you will discover how you can use Keras to develop and evaluate neural network models for multi-class classification problems.After completing this step-by-step tutorial, you will know:- How to load data from CSV and make it available to Keras.
- How to prepare multi-class classification data for modeling with neural networks.
- How to evaluate Keras neural network models with scikit-learn.
- This example demonstrates the use of fasttext for text classification (GitHub repo)
- http://stackoverflow.com/questions/tagged/keras
======GPU and distributed (Keras + TensorFlow)
- Distributed deep learning with Keras and Apache Spark (GitHub repo)
Spark 2.0
If you want to run the examples using Apache Spark 2.0.0 and higher. You will need to remove the line containing sqlContext = SQLContext(sc). We need to do this because in Spark 2.0+, the SQLContext, and Hive context are now merged in the Spark session.
