Published on Thu Oct 15 2020

Multi-Task Learning for Cross-Lingual Abstractive Summarization

Sho Takase, Naoaki Okazaki

We present a multi-task learning framework for cross-lingual abstractive summarization to augment training data. The experimental results show that Transum achieves better performance than the model trained with only pseudo cross- linguistic summarization data.

0
0
0
Abstract

We present a multi-task learning framework for cross-lingual abstractive summarization to augment training data. Recent studies constructed pseudo cross-lingual abstractive summarization data to train their neural encoder-decoders. Meanwhile, we introduce existing genuine data such as translation pairs and monolingual abstractive summarization data into training. Our proposed method, Transum, attaches a special token to the beginning of the input sentence to indicate the target task. The special token enables us to incorporate the genuine data into the training data easily. The experimental results show that Transum achieves better performance than the model trained with only pseudo cross-lingual summarization data. In addition, we achieve the top ROUGE score on Chinese-English and Arabic-English abstractive summarization. Moreover, Transum also has a positive effect on machine translation. Experimental results indicate that Transum improves the performance from the strong baseline, Transformer, in Chinese-English, Arabic-English, and English-Japanese translation datasets.

Fri May 28 2021
NLP
Cross-Lingual Abstractive Summarization with Limited Parallel Resources
Parallel cross-lingual summarization data is scarce. We propose a novel Multi-Task framework for Cross-Lingual Summarization (MCLAS) in a low-resource setting. Employing one unified decoder to generate the sequential concatenation of monolingual and cross-
7
0
1
Sat Aug 31 2019
NLP
NCLS: Neural Cross-Lingual Summarization
Cross-lingual summarization (CLS) is the task to produce a summary in one language for a source document in a different language. Existing methods simply divide this task into two steps: summarization and translation,leading to the problem of error propagation. To handle that,
0
0
0
Thu Sep 09 2021
NLP
ARMAN: Pre-training with Semantically Selecting and Reordering of Sentences for Persian Abstractive Summarization
Arman is a Transformer-based encoder-decoder model pre-trained with three novel objectives. salient sentences from a document are selected according to a modified semantic score to be masked and form a pseudo summary. We evaluated our proposed models on six downstream Persian summarization tasks.
0
0
0
Tue Dec 08 2020
Machine Learning
Cross-lingual Approach to Abstractive Summarization
Automatic text summarization extracts important information from texts and presents the information in the form of a summary. In several natural language processing tasks, cross-lingual model transfers are successfully applied in low-resource languages. In our work, we used a pretrained English summarization model based on
3
0
3
Thu Aug 22 2019
Machine Learning
Text Summarization with Pretrained Encoders
Bidirectional Encoder Representations from Transformers (BERT) represents the latest incarnation of pretrained language models. We introduce a novel document-level encoder based on BERT which is able to express the semantics of a document and obtain representations for its sentences. For abstractive
0
0
0
Thu Feb 18 2021
NLP
Meta-Transfer Learning for Low-Resource Abstractive Summarization
Neural abstractive summarization has been studied in many pieces of literature. It achieves great success with the aid of large corpora. However, one may not always benefit from transfer learning due to the domain shifting problem.
3
0
0