Published on Mon Oct 10 2016

Dynamic Metric Learning from Pairwise Comparisons

Kristjan Greenewald, Stephen Kelley

Online Convex Ensemble StrongLy Adaptive Dynamic Learning (OCELAD) is a general adaptive, online approach for learning and tracking optimal metrics. OCELAD is highly robust to a variety of nonstationary behaviors in the changing metric.

0
0
0
Abstract

Recent work in distance metric learning has focused on learning transformations of data that best align with specified pairwise similarity and dissimilarity constraints, often supplied by a human observer. The learned transformations lead to improved retrieval, classification, and clustering algorithms due to the better adapted distance or similarity measures. Here, we address the problem of learning these transformations when the underlying constraint generation process is nonstationary. This nonstationarity can be due to changes in either the ground-truth clustering used to generate constraints or changes in the feature subspaces in which the class structure is apparent. We propose Online Convex Ensemble StrongLy Adaptive Dynamic Learning (OCELAD), a general adaptive, online approach for learning and tracking optimal metrics as they change over time that is highly robust to a variety of nonstationary behaviors in the changing metric. We apply the OCELAD framework to an ensemble of online learners. Specifically, we create a retro-initialized composite objective mirror descent (COMID) ensemble (RICE) consisting of a set of parallel COMID learners with different learning rates, demonstrate RICE-OCELAD on both real and synthetic data sets and show significant performance improvements relative to previously proposed batch and online distance metric learning algorithms.

Fri Mar 11 2016
Machine Learning
Nonstationary Distance Metric Learning
ComID-SADL is an adaptive, online approach for learning and tracking optimal metrics as they change over time. It is highly robust to a variety of behaviors in the changing metric. It shows significant performance improvements relative to previously proposed batch and online distance metric learning algorithms.
0
0
0
Sat Jan 07 2017
Machine Learning
Similarity Function Tracking using Pairwise Comparisons
Online Convex Ensemble StrongLy Adaptive Dynamic Learning (OCELAD) is a general adaptive, online approach for learning and tracking optimal metrics. We show significant performance improvements and increased robustness to nonstationary effects relative to previously proposedbatch and online distance metric learning algorithms.
0
0
0
Fri Dec 14 2018
Machine Learning
A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms, Experimental Analysis, Prospects and Challenges (with Appendices on Mathematical Background and Detailed Algorithms Explanation)
distance metric learning is a branch of machine learning that aims to learn distances from the data. This tutorial provides a theoretical background and foundations on this topic. It also provides an experimental analysis of the most-known algorithms.
0
0
0
Mon Sep 15 2014
Machine Learning
Active Metric Learning from Relative Comparisons
This work focuses on active learning of distance metrics from relative comparison information. A relative comparison specifies, for a data point, that instance is more similar to $x-j than to $X-k. We present an information-theoretic criterion that
0
0
0
Mon May 12 2014
Artificial Intelligence
Two-Stage Metric Learning
In this paper, we present a novel two-stage metric learning algorithm. We first map each learning instance to a probability distribution. Then, we define the distance in input data space as the Fisher information distance.
0
0
0
Fri May 08 2020
Machine Learning
Project and Forget: Solving Large-Scale Metric Constrained Problems
Project and Forget is an algorithm that uses Bregman projections to solve metric constrained problems with many (possibly exponentially) inequality constraints. We prove that the distance of the current iterate to the optimal solution decays at an exponential rate.
1
0
0