The 5th NTCIR Workshop
Brief Task Description

[Japanese] [NTCIR-5]


The NTCIR is a series of evaluation workshop to enhance the research in information access technologies, including text retrieval, cross-lingual information access, question answering, etc, by providing re-usable test collections, evaluation infrastructures, and a forum of researchers. The emphasis has been placed on, but not limited to, Japanese and other Asian languages, and cross-lingual applications between Asian languages and English. The workshops are periodical events which are held once per about one and a half years (about 18 months).

The detailed task and collection description will be available in each task's web site.


The 5th NTCIR Workshop selected the following 5 areas of research as "Tasks" and 1 area as "pilot task"; For details, please visit each task's call for participation below.

1. Cross-Lingual Information Retrieval Task (CLIR)
Multilingual CLIR
Bilingual CLIR and Pivot Bilingual CLIR, and
Single language IR
Languages: Traditional Chinese, Korean, English and Japanese
2. Cross-Language Question Answering Task (CLQA)
CLQA is a new task based on CLIR and QAC
5 subtasks (C->C, E->C, C->E, E->J, J->E) are provided (J->J will is provided in QAC).
3. Patent Retrieval Task (PATENT)
Retrieval task: "Invalidity search"
Classification task: The purpose is to categorize target patent applications based on the F-term classification system.
4. Question Answering Task (QAC)
In a simulated interactive situation, all and only correct answers should be listed as its response for each question.
Task for reference evaluation: all references are resolved manually and the questions are made understandable in isolation.
5. Web Task (WEB)
Navigational Retrieval Task: known item search
pilot task: Query Term Expansion Task
6. Pilot Workshop: Multimodal Summarization of Trend Information (MuST)@--Japanese only


NTCIR-4 CLIR used Traditional Chinese, Korean, Japanese, English documents published in East Asia in 1998-1999. NTCIR-5 CLIR add Chinese, Korean and Japanese (in 2000-2001) newspaper articles, so the size of the document collections are increased. NTCIR-3 CLIR and NTCIR-4 CLIR, the test collections used in CLIR at NTCIR-3 and NTCIR-4, are provided as training collections (including the document collections).

CLQA uses Chinese, Japanese, English newspaper articles published in 2000-2001.

NTCIR-5 PATENT uses the same test collection as NTCIR-4 PATENT. NTCIR-3 PATENT and NTCIR-4 PATENT, the test collection used in PATENT at NTCIR-3 and NTCIR-4 are available for training (user agreement form is required).

NTCIR-4 QAC used Japanese newspaper articles published in 1998-1999, and the size of the document at NTCIR-5 QAC is increased to 1998-2001.NTCIR-4 QAC, the test collections used in QACR at NTCIR-4, is provided as training collections (including the document collections).

WEB uses NW1000G-04. NTCIR-3 WEB and NTCIR-4 WEB, the test collection used in WEB at NTCIR-3 and NTCIR-4 are available for training (user agreement form is required).


Please consult How to Participate. Online registration form is linked from the "How to Participate"


2004-09-30   Release of User Agreements Forms (PATENT, WEB) Done
2004-22-16   Release of User Agreements Forms (CLIR) Done
  Release of User Agreements Forms (QAC) Done
2004-12-07   Release of User Agreements Dorms (CLQA) Done
2005-01-07    Releae of User Agreements Forms (MuST)  
2005-05-31   Registration Due
(registration for task participation)

ŠParticipation for MuST is still possible).
Each task's evaluation schedule varies.
Please see each task's web site.
2005-09-01    Evaluation Results Release
Call for papers for Open submission paper
2005-10-01   Submission of Extended abstracts of the open submission paper  
2005-10-15   Deadline of manuscript for Proceedings of the Meeting**  
2005-11-01   Acceptance notice or open submission paper  
2005-12-06,07,08,09   Meeting, NII, Tokyo, Japan  
    Task data/test collections will be available for research purpose use  

[Japanese] [NTCIR-5]
This page started on : 2005-12-15