A Study on the Dialogue-based Question Answering System: Focusing on Error Recovery

Abstract

Question answering systems are a computer system that assists a persons cognitive activity by providing a direct answer to the persons question. However, they lack communication abilities, which limits the task of question answering to simply querying the dataset. There are two reasons why question answering systems do not develop into further communication although question answering is apparently a type of communication: first, the systems are not leveraging the rich information they have. Second, they are inflexible and inadaptable as they aim to be a robust system. In fact, it is difficult to guess the intention of a person precisely because natural language has ambiguities at different levels. Apart from the ambiguity of natural language, people have diversity due to social and cultural factors.

Dialogue-based question answering is a way to overcome the limitations of not properly deploying the large data and not coping with the ambiguity in question answering. The paper defined a dialogue-based question answering system as a system that uses communication strategies: (i) to provide the user with information proactively and (ii) to accommodate user reactions. The use of the systems information allows to enrich the question answering conversation, and the use of the users feedback allows to continue the rich conversation coherently.

First, the paper introduced question answering systems and communication theory to construe question answering as a form of communication. Next, it analyzed the WikiQA dataset and presented eleven types of errors which occurred in the actual use of question answering systems but could not be handled by the performance improvement of each module in the systems. After introducing how dialogue can be integrated in question answering systems in terms of extended speech act and the error types, it pilot-tested a question answering chatbot. The result showed that the two communication strategies could be another interface that helps both the user and the system explore huge data spaces by letting them make agreements and recover errors.