This page provides resources about image similarity using deep learning, Siamese network, one-shot learning.
There are several ways to compute image similarity with deep learning.
One can either train an end to end deep model which learns similarity between images, or use the Deep model as a feature extractor and then use a standard similarity metric (Dot product, L2 distance etc.)
1) Using a deep learning model as a feature extractor: This mechanism is very common.
It depends on the problem domain at hand. For example, if these are natural images with scenes/objects, one can use a pre-trained deep model (say, a GoogleNet model trained on ImageNet or maybe Natural scene images (like, say Places-205) and then extract features from one of the last few layers (either fc6 or fc7). One can extract features and then compute a distance between them. This is also used, for example, in Face recognition or Face verification.
2) A more complicated way of doing this, which also might be more principled or end-to-end, is training a similarity network. These models are called Siamese Networks. Obviously for this, you need to have a lot of data to train these models from scratch.
Siamese network is a neural network that contain two or more identical subnetwork. The purpose of this network is to find the similarity or comparing the relationship between two comparable things. Unlike classification task that uses cross entropy as the loss function, siamese network usually uses contrastive loss or triplet loss.
- Implementing Siamese Network using Tensorflow with MNIST (GitHub repo)
A web page (its GitHub repo) motivates the author to implement a similar Siamese network using Tensorflow.
- Using siamese network to do dimensionality reduction and similar image retrieval (GitHub repo)
- Image Similarity using Deep Ranking (GitHub repo, Blog post — PDF)
- Similarity Learning with (or without) Convolutional Neural Network (Lecture Slides, PDF)
(mostly) reimplimented this paper (koch et al, Siamese Networks for one-shot learning) in Keras. Trains on the Omniglot dataset.
Also check out the author’s blog post about the paper and one shot learning.
- A Siamese similarity model for image similarity (GitHub repo using Keras)
- Predicting Image Similarity using Siamese Networks (PDF)
- Image Similarity with Siamese Networks — Kaggle (PDF)
Related Papers:
- Bell, S., & Bala, K. (2015). Learning visual similarity for product design with convolutional neural networks. ACM Transactions on Graphics (TOG), 34(4), 98. (PDF)
- Wang, J., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., & Wu, Y. (2014). Learning fine-grained image similarity with deep ranking. arXiv preprint arXiv:1404.4661. (PDF)
A Deep Ranking algorithm using Saimese CNN seems to achieve great success compared to most algorithms shown in the paper.
References:
