Published on Thu Apr 30 2020

Interpretable Entity Representations through Large-Scale Typing

Yasumasa Onoe, Greg Durrett

In standard methodology for natural language processing, entities in text are typically embedded in dense vector spaces with pre-trained models. The representations produced this way are effective when fed into downstream models, but require end-task fine-tuning and are fundamentally difficult to interpret.

0
0
0
Abstract

In standard methodology for natural language processing, entities in text are typically embedded in dense vector spaces with pre-trained models. The embeddings produced this way are effective when fed into downstream models, but they require end-task fine-tuning and are fundamentally difficult to interpret. In this paper, we present an approach to creating entity representations that are human readable and achieve high performance on entity-related tasks out of the box. Our representations are vectors whose values correspond to posterior probabilities over fine-grained entity types, indicating the confidence of a typing model's decision that the entity belongs to the corresponding type. We obtain these representations using a fine-grained entity typing model, trained either on supervised ultra-fine entity typing data (Choi et al. 2018) or distantly-supervised examples from Wikipedia. On entity probing tasks involving recognizing entity identity, our embeddings used in parameter-free downstream models achieve competitive performance with ELMo- and BERT-based embeddings in trained models. We also show that it is possible to reduce the size of our type set in a learning-based way for particular domains. Finally, we show that these embeddings can be post-hoc modified through a small number of rules to incorporate domain knowledge and improve performance.

Sun Jan 08 2017
Artificial Intelligence
Multi-level Representations for Fine-Grained Typing of Knowledge Base Entities
Entities are essential elements of natural language. We investigate state-of-the-art learning methods on each level. We confirm experimentally that each level of representation contributes complementary information. A joint representation of all three levels improves embedding based baseline by a large margin.
0
0
0
Mon Mar 23 2020
NLP
E2EET: From Pipeline to End-to-end Entity Typing via Transformer-Based Embeddings
0
0
0
Sat Jan 02 2021
NLP
Modeling Fine-Grained Entity Types with Box Embeddings
Neural entity typing models typically represent fine-grained entity types in a high-dimensional space. Such spaces are not well-suited to modeling these types' complex interdependencies. We study the ability of box embeddings, which embed concepts as d-dimensional hyperrectangles.
2
12
55
Mon Aug 07 2017
NLP
Corpus-level Fine-grained Entity Typing
This paper addresses the problem of corpus-level entity typing. The application of entity typing is to learn which classes an entity is a member of. We propose FIGMENT to tackle this problem.
0
0
0
Thu Dec 22 2016
NLP
Noise Mitigation for Neural Entity Typing and Relation Extraction
In this paper, we address two different types of noise in informationExtraction models: noise from distant supervision and noise from pipeline input. Our target tasks are entity typing and relation extraction. Our experiments show that probabilistic predictions are more robust than discrete predictions.
0
0
0
Sun Apr 19 2020
NLP
A Chinese Corpus for Fine-grained Entity Typing
This paper introduces a corpus for Chinese fine-grained entity typing. It contains 4,800mentions manually labeled through crowdsourcing. Each mention is annotated with free-form entity types. We also categorize all the fine- grained types into 10 general types.
0
0
0