The Natural Language Decathlon: Multitask Learning as Question Answering
Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task.
Also cited · not yet reviewed (1)
- Transformer2017 · cited 4דWe provide a set of baselines for decaNLP that combine the basics of sequence-to-sequence learning [Sutskever et al. 2014, Bahdanau et al. 2014, Luong et al. 2015b] with pointer networks [Vinyals et al. 2015, Merity et a…”From this paper · §Introduction
Led to
- T52019 · cited 4×, 2 in Method“This approach is inspired by previous unifying frameworks for NLP tasks, including casting all text problems as question answering (McCann et al. 2018), language modeling (Radford et al. 2019), or span extraction Keskar…”From T5 · §Introduction
- LPAQA (What LMs know)2019 · cited 2דIn previous work (McCann et al. 2018; Radford et al. 2019; Petroni et al. 2019), trt_{r} has been a single manually defined prompt based on the intuition of the experimenter.”From LPAQA (What LMs know) · §Knowledge Retrieval from LMs
- 12-in-12019 · cited 5×, 2 in Method“Inspired by prior multi-task literature bengio2009curriculum mccann2018natural, we experimented with both curriculum and anti-curriculum strategies based on task difficulty.”From 12-in-1 · §Approach
Abstract
Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten tasks: question answering, machine translation, summarization, natural language inference, sentiment analysis, semantic role labeling, zero-shot relation extraction, goal-oriented dialogue, semantic parsing, and commonsense pronoun resolution. We cast all tasks as question answering over a context. Furthermore, we present a new Multitask Question Answering Network (MQAN) jointly learns all tasks in decaNLP without any task-specific modules or parameters in the multitask setting. MQAN shows improvements in transfer learning for machine translation and named entity recognition, domain adaptation for sentiment analysis and natural language inference, and zero-shot capabilities for text classification. We demonstrate that the MQAN's multi-pointer-generator decoder is key to this success and performance further improves with an anti-curriculum training strategy. Though designed for decaNLP, MQAN also achieves state of the art results on the WikiSQL semantic parsing task in the single-task setting. We also release code for procuring and processing data, training and evaluating models, and reproducing all experiments for decaNLP.