Published on Thu Oct 11 2018

Classification using margin pursuit

Matthew J. Holland

The classical approach to this problem is simply maximization of the expected margin. We propose an algorithm which searches the hypothesis space in such a way that a pre-set "margin level" ends up being a robust estimator of the margin location.

0
0
0
Abstract

In this work, we study a new approach to optimizing the margin distribution realized by binary classifiers. The classical approach to this problem is simply maximization of the expected margin, while more recent proposals consider simultaneous variance control and proxy objectives based on robust location estimates, in the vein of keeping the margin distribution sharply concentrated in a desirable region. While conceptually appealing, these new approaches are often computationally unwieldy, and theoretical guarantees are limited. Given this context, we propose an algorithm which searches the hypothesis space in such a way that a pre-set "margin level" ends up being a distribution-robust estimator of the margin location. This procedure is easily implemented using gradient descent, and admits finite-sample bounds on the excess risk under unbounded inputs. Empirical tests on real-world benchmark data reinforce the basic principles highlighted by the theory, and are suggestive of a promising new technique for classification.

Wed Jan 23 2019
Machine Learning
Large dimensional analysis of general margin based classification methods
Margin-based classifiers have been popular in both machine learning andistics for classification problems. Since a large number of classifiers are available, one natural question is which type of classifier should be used given a particular task. We aim to answer this question by investigating the
0
0
0
Fri Apr 18 2008
Machine Learning
Margin-adaptive model selection in statistical learning
A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one.
0
0
0
Tue Aug 13 2019
Machine Learning
Comparison theorems on large-margin learning
This paper studies binary classification problem associated with a family of loss functions called large-margin unified machines (LUM) It offers anatural bridge between distribution-based likelihood approaches and margin-based approaches.
0
0
0
Mon Jul 06 2015
Machine Learning
A Simple Algorithm for Maximum Margin Classification, Revisited
In this note, we revisit the algorithm of Har-Peled et. al. for computing a linear maximum margin classifier. Our presentation is self contained, and the algorithm itself is slightly simpler than the original algorithm.
0
0
0
Thu Oct 29 2020
Machine Learning
The Performance Analysis of Generalized Margin Maximizer (GMM) on Separable Data
Logistic models are commonly used for binary classification tasks. The success of such models has often been attributed to their connection to maximum-likelihood estimators. In this paper, we present and study a more general setting, where the underlying parameters of the logistic model possess certain structures (sparse, low-rank, etc.)
0
0
0
Thu Jul 16 2020
Machine Learning
Large scale analysis of generalization error in learning using margin based classification methods
0
0
0