NTCIR Project
NTCIR-5 QAC
Research Purpose Use of Test Collection

[JAPANESE] [NTCIR Home] [NTCIR DATA Home]


NTCIR-5 QAC@(Q&A data Test Collection)

Test Collection
The collection consists of
@@@@* Document data(the Mainichi News Paper articles 2000-2001 and the Yomiuri News Paper articles 2000-2001)
@@@@* Q&A data set(There are 360 queries in Japanese, which constitue 50 series) and answers
@@@@* a scoring tool
Researchers can get the date set which includes the Q&A data set and a scoring tool from NII.

Collection Task Documents Task data
Genre Filename Lang Year # of docs Size Topic/ Relevanc judge  
Lang. #

NTCIR-5 QA

QAC3

News Paper articles

ntc5-j01-mai00.txt

J

2000

about120000

about135Mb

J

queries


2 (3)


ntc5-j01-mai01.txt

2001

about110000

about143Mb

ntc5-j01-yomi00.txt

2000

about130000

about183Mb

ntc5-j01-yomi01.txt

2001

about240000

about312Mb

– archers can get the date set which includes the Q&A data set and a scoring tool from NII.But you should obatain the document data as follow.

Documents, Topics and Questions

@@

Two kinds of document data should be used as follow: Please refer to the page" How to obtain Newspaper Article Data"

Task

  1. The task setting follows subtask 3 in NTCIR-4 QAC. That is, participants are requested to answer series of questions in a simulated interactive information gathering for report writing.

  2. For each question, all and only correct answers should be listed as its response.

  3. In each series of questions, the first question is marked and other questions have references to portions of or answers to some of the previous questions.

  4. The participants must not look ahead to the questions following the one currently being handled. This restriction reflects the fact that the task is a simulation of interactive use of QA systems in dialogues.

  5. Answers are names and values extracted from target documents. Those include named entities such as names of persons and organizations, numerical expressions such as monetary and metrological values, titles, dates, and names of species and categories. In addition, event type description in compound nouns and conventional constructions for round numbers and ranges are also included.

Question File Format

The Question File consists of lines with the following format.

[QID]: "[QUESTION]"<CR>

Answer File Format

The Answer File consists of lines with the following format (so called CSV format).

The followings are the procedures to obtain this NTCIR-5 QAC test collection. The test collection and data available from NII are free of charge.


Reference

The terms of use [PDF]
README[txt]
Task Overview of NTCIR 5 QAC
An Overview of NTCIR-5 QAC3

Contact Us

Email: ntc-secretariat

Mailing List

The release of the new test collections and correction information shall be announced through the ntcir Mailing list

Notice

The test collection has been constructed and used for the NTCIR. They are usable only for the research purpose use.
The documents collection included in the test collection were provided to NII for used in NTCIR free of charge or for a fee. The providers of the document data kindly understand the importance of the test collection in the research on information access technologies and then granted the use of the data for research purpose. Please remember that the document data in the NTCIR test collection is copyrighted and has commercial value as data. It is important for our continued reliable and good relationship with the data producers/providers that we researchers must behave as a reliable partners and use the data only for research purpose under the user agreement and use them carefully not to violate any rights for them .