CoQA: A Conversational Question Answering Challenge
Humans gather information by engaging in conversations involving a series of interconnected questions and answers. For machines to assist in information gathering, it is therefore essential to enable them to answer conversational questions.
Also cited · not yet reviewed (3)
- DrQA2017 · cited 3×, 2 in Method“We use the Document Reader (DrQA) model of Chen et al. 2017, which has demonstrated strong performance on multiple datasets Rajpurkar et al. 2016; Labutov et al. 2018.”From this paper · §Models
- SQuAD2016 · cited 3×, 1 in Method“We use the Document Reader (DrQA) model of Chen et al. 2017, which has demonstrated strong performance on multiple datasets Rajpurkar et al. 2016; Labutov et al. 2018.”From this paper · §Models
- Bahdanau attention2014 · cited 1×, 1 in Method“Motivated by their success, we use a sequence-to-sequence with attention model for generating answers Bahdanau et al. 2015.”From this paper · §Models
Led to
- UniLM2019 · cited 5דWe conduct experiments on the Stanford Question Answering Dataset (SQuAD) 2.0 [34], and Conversational Question Answering (CoQA) [35] datasets.”From UniLM · §Experiments
- BoolQ2019 · cited 2דYes/No questions make up a subset of the reading comprehension datasets CoQA Reddy et al. 2018, QuAC Choi et al. 2018, and HotPotQA Yang et al. 2018, and are present in the ShARC Saeidi et al. 2018 dataset.”From BoolQ · §Related Work
- GPT-32020 · cited 2דAs fine-tuned language models have neared human performance on many standard benchmark tasks, considerable effort has been devoted to constructing more difficult or open-ended tasks, including question answering [58, 47,…”From GPT-3 · §Related Work
- BIG-bench2022 · cited 2דTask cites: (Reddy et al. 2019; Radford et al. 2019; Brown et al. 2020)”From BIG-bench · §Author contributions
Abstract
Humans gather information by engaging in conversations involving a series of interconnected questions and answers. For machines to assist in information gathering, it is therefore essential to enable them to answer conversational questions. We introduce CoQA, a novel dataset for building Conversational Question Answering systems. Our dataset contains 127k questions with answers, obtained from 8k conversations about text passages from seven diverse domains. The questions are conversational, and the answers are free-form text with their corresponding evidence highlighted in the passage. We analyze CoQA in depth and show that conversational questions have challenging phenomena not present in existing reading comprehension datasets, e.g., coreference and pragmatic reasoning. We evaluate strong conversational and reading comprehension models on CoQA. The best system obtains an F1 score of 65.4%, which is 23.4 points behind human performance (88.8%), indicating there is ample room for improvement. We launch CoQA as a challenge to the community at http://stanfordnlp.github.io/coqa/