Published on Sun Jan 21 2018

A Survey of Word Embeddings Evaluation Methods

Amir Bakarov

Word embeddings are real-valued word representations able to capture lexical semantics and trained on natural language corpora. Models proposing these representations have gained popularity in the recent years. The issue of the most adequate evaluation method still remains open.

0
0
0
Abstract

Word embeddings are real-valued word representations able to capture lexical semantics and trained on natural language corpora. Models proposing these representations have gained popularity in the recent years, but the issue of the most adequate evaluation method still remains open. This paper presents an extensive overview of the field of word embeddings evaluation, highlighting main problems and proposing a typology of approaches to evaluation, summarizing 16 intrinsic methods and 12 extrinsic methods. I describe both widely-used and experimental methods, systematize information about evaluation datasets and discuss some key challenges.

Mon Jan 28 2019
NLP
Evaluating Word Embedding Models: Methods and Experimental Results
Extensive evaluation on a large number of word embedding models is conducted. Intrinsic evaluators test the quality of a representation independent of specific natural language processing tasks. Extrinsics use word embeddings as input features to a downstream task.
0
0
0
Tue Feb 07 2017
NLP
How to evaluate word embeddings? On importance of data efficiency and simple supervised tasks
The single most important goal of representation learning is making subsequent learning faster. Surprisingly, this fact is not well reflected in the way embeddings are evaluated. We argue that focus of word representation evaluation should reflect those trends.
0
0
0
Sun May 08 2016
NLP
Problems With Evaluation of Word Embeddings Using Word Similarity Tasks
The NLP community has relied heavily on word similarity tasks as a proxy for intrinsic evaluation of word vectors. Word similarity evaluation is attractive because it is computationally inexpensive and fast. Our study suggests that the use of word similarity task is not sustainable and calls for further research on evaluation methods.
0
0
0
Tue Jan 15 2013
Machine Learning
The Expressive Power of Word Embeddings
We seek to better understand the difference in quality of the several publicly released embeddings. Our evaluation of sentiment polarization and synonym/antonym relations shows that embeddings are able to capture surprisingly nuanced semantics even in the absence of sentence structure.
0
0
0
Fri Jan 19 2018
NLP
Size vs. Structure in Training Corpora for Word Embedding Models: Araneum Russicum Maximum and Russian National Corpus
In this paper, we present a distributional word embedding model trained on one of the largest available Russian corpora: Araneum Russicum Maximum. We compare this model to the model trained on the Russian National Corpus. The two corpora are much different in their size and compilation procedures.
0
0
0
Wed Apr 11 2018
NLP
Evaluating Word Embedding Hyper-Parameters for Similarity and Analogy Tasks
The impact of hyper-parameters when training an embedding model is often poorly understood. How much do vector dimensions and corpus size affect the quality of embeddings? And how do these results translate to downstream applications?
0
0
0