Minimal Loss Hashing for Compact Binary Codes Mohammad Norouzi David Fleet University of Toronto

Near Neighbor Search

Near Neighbor Search

Near Neighbor Search

Similarity-Preserving Binary Hashing Why binary codes?

 Sub-linear search using hash indexing (even exhaustive linear search is fast)  Binary codes are storage-efficient

Similarity-Preserving Binary Hashing Hash function binary quantization parameter matrix input vector

kth

row of

W

Random projections used by

locality-sensitive hashing

(LSH) and related techniques

[Indyk & Motwani ‘98; Charikar ’02; Raginsky & Lazebnik ’09]

Learning Binary Hash Functions Reasons to learn hash functions:  to find more compact binary codes  to preserve general similarity measures Previous work  boosting

[Shakhnarovich et al ’03]

 neural nets

[Salakhutdinov & Hinton 07; Torralba et al 07]

 spectral methods

[Weiss et al ’08]

 loss-based methods

[Kulis & Darrel ‘09]

 …

Formulation Input data: Similarity labels: Binary codes: Hash function:

Loss Function Hash code quality measured by a loss function: binary : code for item 1 : code for item 2 : similarity label similarity label cost measures consistency Similar items should map to nearby hash codes Dissimilar items should map to very different codes

Hinge Loss Similar items should map to codes within a radius of bits Dissimilar items should map to codes no closer than bits

Empirical Loss Given training pairs with similarity labels Good:  incorporates quantization and Hamming distance Not so good:  discontinuous, non-convex objective function

We minimize an upper bound on empirical loss, inspired by structural SVM formulations

[Taskar et al ‘03; Tsochantaridis et al ‘04; Yu & Joachims ‘09]

Bound on loss

LHS = RHS

Bound on loss

Remarks:

   piecewise linear in

W

convex-concave in

W

relates to structural SVM with latent variables

[Yu & Joachims ‘09]

Bound on Empirical Loss Loss-adjusted inference  Exact  Efficient

Perceptron-like Learning  Initialize with LSH  Iterate over pairs • Compute , the codes given by • Solve loss-adjusted inference • Update

[McAllester et al.., 2010]

Experiment: Euclidean ANN Similarity based on Euclidean distance Datasets  LabelMe (GIST)      MNIST (pixels) PhotoTourism (SIFT) Peekaboom (GIST) Nursery (8D attributes) 10D Uniform

Experiment: Euclidean ANN 22K LabelMe  512 GIST  20K training  2K testing  ~1% of pairs are similar Evaluation  Precision: #hits / number of items retrieved  Recall: #hits / number of similar items

Techniques of interest  MLH – minimal loss hashing (This work)  LSH – locality-sensitive hashing (Charikar ‘02)  SH – spectral hashing (Weiss, Torralba & Fergus ‘09)  SIKH – shift-Invariant kernel hashing (Raginsky & Lazebnik ‘09)  BRE – Binary reconstructive embedding (Kulis & Darrel ‘09)

Euclidean Labelme – 32 bits

Euclidean Labelme – 32 bits

Euclidean Labelme – 32 bits

Euclidean Labelme – 64 bits

Euclidean Labelme – 64 bits

Euclidean Labelme – 128 bits

Euclidean Labelme – 256 bits

Experiment: Semantic ANN Semantic similarity measure based on annotations (object labels) from LabelMe database:  512D GIST, 20K training, 2K testing Techniques of interest  MLH – minimal loss hashing  NN – nearest neighbor in GIST space  NNCA – multilayer network with RBM pre-training and nonlinear NCA fine tuning

[Torralba , et al. ’09; Salakhutdinov & Hinton ’07]

Semantic LabelMe

Semantic LabelMe

Summary A formulation for learning binary hash functions based on   structured prediction with latent variables hinge-like loss function for similarity search Experiments show that with minimal loss hashing   binary codes can be made more compact semantic similarity based on human labels can be preserved

Thank you!

Questions?