Published on Wed Jan 22 2020

Optimal estimation of sparse topic models

Xin Bing, Florentina Bunea, Marten Wegkamp

This paper studies the estimation of that is possibly element-wise sparse. The number of topics is unknown. We derive a finite sample upper bound for our estimator, and show that it matches the minimax lower bound in many scenarios.

0
0
0
Abstract

Topic models have become popular tools for dimension reduction and exploratory analysis of text data which consists in observed frequencies of a vocabulary of words in documents, stored in a matrix. The main premise is that the mean of this data matrix can be factorized into a product of two non-negative matrices: a word-topic matrix and a topic-document matrix . This paper studies the estimation of that is possibly element-wise sparse, and the number of topics is unknown. In this under-explored context, we derive a new minimax lower bound for the estimation of such and propose a new computationally efficient algorithm for its recovery. We derive a finite sample upper bound for our estimator, and show that it matches the minimax lower bound in many scenarios. Our estimate adapts to the unknown sparsity of and our analysis is valid for any finite , , and document lengths. Empirical results on both synthetic data and semi-synthetic data show that our proposed estimator is a strong competitor of the existing state-of-the-art algorithms for both non-sparse and sparse , and has superior performance is many scenarios of interest.