| The 3rd NTCIR Workshop was OVER. |
For the current NTCIR Workshop (the latest in the series of NTCIR Workshops), visit
 ! |
The Third NTCIR Woorkshop QA task
Question Answering Challenge (QAC)
| The 3rd NTCIR Workshop was OVER. |
| For the current NTCIR Workshop (the latest in the series of NTCIR Workshops), visit
! |
last update July 30, 2001
Junichi Fukumoto
Department of Computer Science, Ritsumeikan University
Postal Address: 1-1-1 Nojihigashi, Kusatsu-shi, Shiga 525-8577, JAPAN
Fax: +81-77-561-2669
E-mail: fukumoto
Tsuneaki Kato
Department of Language and Information Sciences, Tokyo University
Postal Address: 3-8-1 Komaba, Meguro-ku, Tokyo 153-8902, JAPAN
Fax: +81-3-5454-4329
E-mail: kato
Fumito Masui
Department of Information Engineering, Faculty of Engineering, Mie University
Postal Address: 1515 Kamihamacho, Tsu-shi, Mie 514-8507, JAPAN
Fax: +81-59-236-0029
E-mail: masui
Question Answering Challenge (QAC) will be carried out as a sub-task of NTCIR3 Workshop scheduled in October 2002. In QAC, we aimed to encourage the development of practical QA systems in a general domain and focus on research of user interaction and information extraction. User interaction technology leads to actual interaction between computer and person. In actual Q&A between people, there will typically be several interactions in order to confirm the intention of the questions and so on. Information extraction that works in general domain is also an important technology in order to realize real QA system.
Mainichi Newspaper articles in 1998 and 1999
We are planning to have three kinds of QAC tasks as follows:
System extracts five answers from the documents in some order. The inverse number of the order is the score of the question. For example, if the second answer is correct, the score will be 1/2. The highest score will be the score of the question. If there are several correct answers, system will return one of them. There are 100 questions in Task 1. Also, system is required to return support information for each answer of the questions. We assume the support information as a paragraph, 100 letter passage or document which includes the answer. (We have not defined the style of the support information.)
System extracts only one answer from the documents. If the system's answer is correct, the score will be given. If there are several answers, system has to return all the answer. There are 100 questions in Task 2. Also, system is required to return support information for each answer of the questions like Task 1.
This task is the evaluation of a series of questions. The related questions are given for the 30 of questions of Task 2. There will be ellipsis or pronominalized elements in the related questions. There are 30 questions in Task 3. Also, system is required to return support information for each answer of the questions like Task 1.
DATE SCHEDULE NOTE 2001-06-25 Task Plan Meeting: TSC2, QAC finished 2001-08-01 Formal CFP for NTCIR WS 3 2001-09-30 Application Due 2001-10 Round-Table Discussion 2001-12 Dry Run 2002-1 Round-Table Discussion 2002-4 Formal Run 2002-07-01 Evaluation Results Delivery 2002-08-20 Paper for Working Note Due 2002-10 NTCIR Workshop 3 Meeting 2002-12-01 Paper for Final Proceedings Due
Please refer QAC home page http://www.nlp.cs.ritsumei.ac.jp/qac/ to get the newest information.
[Japanese][NTCIR][workshop][register][agreement] Task [clir][patent][qac][tsc][web]