Published on Fri Dec 22 2017

Diversifying Support Vector Machines for Boosting using Kernel Perturbation: Applications to Class Imbalance and Small Disjuncts

Shounak Datta, Sayak Nag, Sankha Subhra Mullick, Swagatam Das

The diversification (generating slightly varying separating discriminators) of Support Vector Machines (SVMs) for boosting has proven to be a challenge due to the strong learning nature of SVMs. Based on the insight that perturbing the SVM kernel may help in diversifying SVMs,

0
0
0
Abstract

The diversification (generating slightly varying separating discriminators) of Support Vector Machines (SVMs) for boosting has proven to be a challenge due to the strong learning nature of SVMs. Based on the insight that perturbing the SVM kernel may help in diversifying SVMs, we propose two kernel perturbation based boosting schemes where the kernel is modified in each round so as to increase the resolution of the kernel-induced Reimannian metric in the vicinity of the datapoints misclassified in the previous round. We propose a method for identifying the disjuncts in a dataset, dispelling the dependence on rule-based learning methods for identifying the disjuncts. We also present a new performance measure called Geometric Small Disjunct Index (GSDI) to quantify the performance on small disjuncts for balanced as well as class imbalanced datasets. Experimental comparison with a variety of state-of-the-art algorithms is carried out using the best classifiers of each type selected by a new approach inspired by multi-criteria decision making. The proposed method is found to outperform the contending state-of-the-art methods on different datasets (ranging from mildly imbalanced to highly imbalanced and characterized by varying number of disjuncts) in terms of three different performance indices (including the proposed GSDI).

Tue Apr 27 2021
Machine Learning
Robust Classification via Support Vector Machines
0
0
0
Sun Aug 30 2015
Machine Learning
Feature Selection via Binary Simultaneous Perturbation Stochastic Approximation
Feature selection (FS) has become an indispensable task in dealing with highly complex pattern recognition problems. In this study, we propose a new wrapper approach for FS based on simultaneous perturbation stochastic approximation (BSPSA)
0
0
0
Tue Oct 16 2018
Machine Learning
An empirical evaluation of imbalanced data strategies from a practitioner's point of view
This research tested the following well known strategies to deal with binary data. The best strategy strongly depends on the metric used to measure the classifier. For AUC and accuracy class weight and the baseline perform better. For F-measure and MCC, SMOTE performs better.
0
0
0
Mon Apr 03 2017
Machine Learning
Geometric Insights into Support Vector Machine Behavior using the KKT Conditions
The support vector machine (SVM) is a powerful and widely used classification algorithm. This paper uses the Karush-Kuhn-Tucker conditions to provide rigorous mathematical proof for new insights into SVM.
0
0
0
Mon Aug 24 2015
Machine Learning
Another Look at DWD: Thrifty Algorithm and Bayes Risk Consistency in RKHS
distance weighted discrimination (DWD) is a margin-based classifier. DWD was originally proposed as a superior alternative to the support vector machine (SVM) However DWD is yet to be popular compared with the SVM.
0
0
0
Wed Sep 03 2014
Machine Learning
Breakdown Point of Robust Support Vector Machine
The support vector machine (SVM) is one of the most successful learning methods for solving classification problems. Despite its popularity, SVM has a serious drawback, that is sensitivity to outliers in training samples. To deal with outliers, robust variants of SVM have been proposed.
0
0
0