NTCIR Project
NTCIR-9 1CLICK (1CLICK-1)
Research Purpose Use of Test Collection

[JAPANESE] [NTCIR Home] [NTCIR Data Home]

NTCIR-9 1CLICK (1CLICK-1)

　　

The NTCIR-9 1CLICK (1CLICK-1) Test Collection was the test collection used at the NTCIR-9 1CLICK (Once Click Access) task.

The NTCIR-9 1CLICK task was: given a Japanese query, return a 500-character or 140-character textual output that aims to satisfy the user as quickly as possible. Important pieces of information should be prioritized and the amount of text the user has to read should be minimized.

The 1CLICK-1 collection contains 60 Japanese queries (15 CELEBRITY + 15 LOCAL + 15 DEFINITION + 15 Question Answering),　each with a set of nuggets.

A nugget represents an atomic piece of factual information, and is a 5-tuple:
- NuggetID
- Nugget weight (importance)
- Nugget semantics (the meaning it conveys)
- Vital string (approximates the minimal length required to convey the meaning of the nugget)
- Source URLs

To evaluate a system output, the system output and the nugget list need to be manually compared using the nugget match interface. Then, using the NTCIREVAL evaluation tool, measures such as S-measure and weighted nugget recall can be computed.

More details, including the test collection statistics, can be found at the 1CLICK-1 homepage
http://research.microsoft.com/en-us/people/tesakai/1click.aspx

and the NTCIR-9 1CLICK Task Overview:
http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings9/NTCIR/01-NTCIR9-OV-1CLICK-SakaiT.pdf

 

The test collection and data are available from NII free of charge.

Reference


Contact us: ntc-secretariat