Published on Tue Aug 17 2021

Incremental cluster validity index-guided online learning for performance and robustness to presentation order

Leonardo Enzo Brito da Silva, Nagasharath Rayapati, Donald C. Wunsch II

iCVI-TopoARTMAP is the first adaptive resonance theory (ART)-based model that uses iCVIs for unsupervised and semi-supervised online learning. The model achieves improved accuracy and robustness to ordering effects by integrating an online iCVI framework as module B of a topological adaptiveResonance Theory predictive mapping.

1
0
0
Abstract

In streaming data applications incoming samples are processed and discarded, therefore, intelligent decision-making is crucial for the performance of lifelong learning systems. In addition, the order in which samples arrive may heavily affect the performance of online (and offline) incremental learners. The recently introduced incremental cluster validity indices (iCVIs) provide valuable aid in addressing such class of problems. Their primary use-case has been cluster quality monitoring; nonetheless, they have been very recently integrated in a streaming clustering method to assist the clustering task itself. In this context, the work presented here introduces the first adaptive resonance theory (ART)-based model that uses iCVIs for unsupervised and semi-supervised online learning. Moreover, it shows for the first time how to use iCVIs to regulate ART vigilance via an iCVI-based match tracking mechanism. The model achieves improved accuracy and robustness to ordering effects by integrating an online iCVI framework as module B of a topological adaptive resonance theory predictive mapping (TopoARTMAP) -- thereby being named iCVI-TopoARTMAP -- and by employing iCVI-driven post-processing heuristics at the end of each learning step. The online iCVI framework provides assignments of input samples to clusters at each iteration in accordance to any of several iCVIs. The iCVI-TopoARTMAP maintains useful properties shared by ARTMAP models, such as stability, immunity to catastrophic forgetting, and the many-to-one mapping capability via the map field module. The performance (unsupervised and semi-supervised) and robustness to presentation order (unsupervised) of iCVI-TopoARTMAP were evaluated via experiments with a synthetic data set and deep embeddings of a real-world face image data set.

Sat Aug 22 2020
Neural Networks
iCVI-ARTMAP: Accelerating and improving clustering using adaptive resonance theory predictive mapping and incremental cluster validity indices
iCVI-ARTMAP is an adaptive resonance theory predictive mapping (ARTMAP) model. It uses incremental cluster validity indices (iCVIs) to perform unsupervised learning. It can achieve running times up to two orders of magnitude shorter than batch CVI computations.
0
0
0
Wed Nov 28 2018
Neural Networks
Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence
This paper presents a novel adaptive resonance theory (ART)-based modular architecture for unsupervised learning. DDVFA consists of a global ART system whose nodes are local fuzzy ART modules. It is equipped with the distinctive features of distributed higher-order activation and match functions.
0
0
0
Fri Oct 18 2019
Machine Learning
Implicit Context-aware Learning and Discovery for Streaming Data Analytics
The performance of machine learning model can be further improved if contextual cues are provided as input along with base features that are directly related to an inference task. In offline learning, one can inspect historical training data to identify contextual clusters. Online training for streaming data may be more challenging.
0
0
0
Mon Jan 25 2021
Machine Learning
Online Continual Learning in Image Classification: An Empirical Survey
Online continual learning for image classification studies the problem of forgetting old tasks in the presence of more recent tasks. Many methods and tricks have been introduced to address this problem, but many have not been fairly and systematically compared under a variety of realistic and practical settings. iCaRL remains competitive when the memory buffer is small; GDumb outperforms many recently proposed methods in medium-size datasets and
4
1
0
Thu Sep 03 2020
Machine Learning
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning
Current deep learning research is dominated by benchmark evaluation. It is assumed that during deployment a model is guaranteed to encounter data that stems from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown instances.
2
64
266
Wed Sep 02 2020
Computer Vision
Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams
Learning prototypes online from streaming data proves a challenging endeavor. We introduce a system where pseudo-prototypes evolve continually in a shared latent space. We obtain state-of-the-art performance by a significant margin on eight benchmarks.
1
2
12
Tue Mar 21 2017
Computer Vision
Simple Online and Realtime Tracking with a Deep Association Metric
Simple Online and Realtime Tracking (SORT) is a pragmatic approach to multiple object tracking with a focus on simple, effective algorithms. In this paper, we integrate appearance information to improve the performance of SORT.
2
0
6
Mon Apr 11 2016
Computer Vision
Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
Face detection and alignment in unconstrained environment are challenging due to various poses, illuminations and occlusions. Recent studies show that deep learning approaches can achieve impressive performance on these two tasks.
0
0
0
Mon Jan 27 2020
Machine Learning
Uncertainty-based Modulation for Lifelong Learning
Algorithm inspired by neuromodulatory mechanisms in the human brain that expands upon Stephen Grossberg\'s ground-breaking Adaptive Resonance Theory proposals. Algorithm is capable of continuous learning of new tasks and under changed conditions with high classification accuracy.
0
0
0
Mon Jan 02 2012
Machine Learning
Scikit-learn: Machine Learning in Python
Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms. Emphasis is put on ease of use,performance, documentation, and API consistency.
0
0
0
Thu Oct 01 2020
Computer Vision
StreamSoNG: A Soft Streaming Classification Approach
Incoming data is normally assigned a crisp label (into one of the structures) and that structure's footprint is incrementally updated. In this paper, we propose a new streaming classification algorithm that uses Neural Gas prototypes.
1
0
0
Sat Aug 22 2020
Neural Networks
iCVI-ARTMAP: Accelerating and improving clustering using adaptive resonance theory predictive mapping and incremental cluster validity indices
iCVI-ARTMAP is an adaptive resonance theory predictive mapping (ARTMAP) model. It uses incremental cluster validity indices (iCVIs) to perform unsupervised learning. It can achieve running times up to two orders of magnitude shorter than batch CVI computations.
0
0
0