[Japanese][NTCIR][workshop][register][agreement] Task [clir][patent][qac][tsc][web]

The Third NTCIR Woorkshop QA task
Question Answering Challenge (QAC)

The 3rd NTCIR Workshop was OVER.
For the current NTCIR Workshop (the latest in the series of NTCIR Workshops), visit  Go NTCIR Workshop ! 

last update July 30, 2001


Junichi Fukumoto
Department of Computer Science, Ritsumeikan University
Postal Address: 1-1-1 Nojihigashi, Kusatsu-shi, Shiga 525-8577, JAPAN
Fax: +81-77-561-2669
E-mail: fukumoto

Tsuneaki Kato
Department of Language and Information Sciences, Tokyo University
Postal Address: 3-8-1 Komaba, Meguro-ku, Tokyo 153-8902, JAPAN
Fax: +81-3-5454-4329
E-mail: kato

Fumito Masui
Department of Information Engineering, Faculty of Engineering, Mie University
Postal Address: 1515 Kamihamacho, Tsu-shi, Mie 514-8507, JAPAN
Fax: +81-59-236-0029
E-mail: masui


Question Answering Challenge (QAC) will be carried out as a sub-task of NTCIR3 Workshop scheduled in October 2002. In QAC, we aimed to encourage the development of practical QA systems in a general domain and focus on research of user interaction and information extraction. User interaction technology leads to actual interaction between computer and person. In actual Q&A between people, there will typically be several interactions in order to confirm the intention of the questions and so on. Information extraction that works in general domain is also an important technology in order to realize real QA system.

Description of Document for QAC

Mainichi Newspaper articles in 1998 and 1999

Outline of QAC task and evlauation

We are planning to have three kinds of QAC tasks as follows:

Task 1

System extracts five answers from the documents in some order. The inverse number of the order is the score of the question. For example, if the second answer is correct, the score will be 1/2. The highest score will be the score of the question. If there are several correct answers, system will return one of them. There are 100 questions in Task 1. Also, system is required to return support information for each answer of the questions. We assume the support information as a paragraph, 100 letter passage or document which includes the answer. (We have not defined the style of the support information.)

Task 2

System extracts only one answer from the documents. If the system's answer is correct, the score will be given. If there are several answers, system has to return all the answer. There are 100 questions in Task 2. Also, system is required to return support information for each answer of the questions like Task 1.

Task 3

This task is the evaluation of a series of questions. The related questions are given for the 30 of questions of Task 2. There will be ellipsis or pronominalized elements in the related questions. There are 30 questions in Task 3. Also, system is required to return support information for each answer of the questions like Task 1.


2001-06-25 Task Plan Meeting: TSC2, QAC finished
2001-08-01 Formal CFP for NTCIR WS 3
2001-09-30 Application Due
2001-10 Round-Table Discussion
2001-12 Dry Run
2002-1 Round-Table Discussion
2002-4 Formal Run
2002-07-01 Evaluation Results Delivery
2002-08-20 Paper for Working Note Due
2002-10 NTCIR Workshop 3 Meeting
2002-12-01 Paper for Final Proceedings Due

URL address of QAC home page

Please refer QAC home page http://www.nlp.cs.ritsumei.ac.jp/qac/ to get the newest information.

[Japanese][NTCIR][workshop][register][agreement] Task [clir][patent][qac][tsc][web]