Published on Wed Feb 10 2021

Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels

Zhaowei Zhu, Yiwen Song, Yang Liu

The knowledge of the label noise transition matrix is crucial to popular solutions to learning with noisy labels. Existing works heavily rely on the existence of "anchor points" or their approximates. Compared with methods using anchor points, our approach uses substantially more instances and benefits from a much better sample

0
0
0
Abstract

The knowledge of the label noise transition matrix, characterizing the probabilities of a training instance being wrongly annotated, is crucial to designing popular solutions to learning with noisy labels, including loss correction and loss reweighting approaches. Existing works heavily rely on the existence of "anchor points" or their approximates, defined as instances that belong to a particular class almost surely. Nonetheless, finding anchor points remains a non-trivial task, and the estimation accuracy is also often throttled by the number of available anchor points. In this paper, we propose an alternative option to the above task. Our main contribution is the discovery of an efficient estimation procedure based on a clusterability condition. We prove that with clusterable representations of features, using up to third-order consensuses of noisy labels among neighbor representations is sufficient to estimate a unique transition matrix. Compared with methods using anchor points, our approach uses substantially more instances and benefits from a much better sample complexity. We demonstrate the estimation accuracy and advantages of our estimates using both synthetic noisy labels (on CIFAR-10/100) and real human-level noisy labels (on Clothing1M and our self-collected human-annotated CIFAR-10).

Sat Jan 11 2020
Machine Learning
Confidence Scores Make Instance-dependent Label-noise Learning Possible
0
0
0
Wed Mar 06 2019
Machine Learning
Safeguarded Dynamic Label Regression for Generalized Noisy Supervision
Latent Class-Conditional Noise model (LCCN) embeds the noise transition under a Bayesian framework. LCCN is constrained on a simplex based on the whole dataset, not some ad-hoc parametric space.
0
0
0
Sat Mar 13 2021
Machine Learning
Learning with Feature-Dependent Label Noise: A Progressive Approach
0
0
0
Wed Dec 02 2020
Machine Learning
Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels
The traditional transition matrix is of vital importance to model label noise and design statistically consistent classifiers. We focus on learning under the conditions of mixed closed-set and open-set label noise. Comprehensive synthetic and real experiments validate that our method can better model the mixed label noise, following its more robust performance.
0
0
0
Thu Dec 10 2020
Machine Learning
Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise
0
0
0
Sat Jun 01 2019
Machine Learning
Are Anchor Points Really Indispensable in Label-Noise Learning?
In label-noise learning, a transition matrix plays a central role in building consistent classifiers. Existing theories have shown that the transition matrix can be learned by exploiting data points that belong to a specific class almost surely. When there are no anchor points, the matrix will be poorly learned.
0
0
0