The 4th NTCIR Workshop
Brief Task Description

[Japanese] [NTCIR-4]
Tasks -- Test Collections -- How to Participate -- NTCIR-4 Meeting & Proceedings NEW!-- Official Language -- DatesNEW!


The NTCIR is a series of evaluation workshop to enhance the research in information access technologies, including text retrieval, cross-lingual information access, text summarization question answering, etc, by providing re-usable test collections, evaluation infrastructures, and a forum of researchers. The emphasis has been placed on, but not limited to, Japanese and other Asian languages, and cross-lingual applications between Asian languages and English. The workshops are periodical events which are held once per about one and a half years (about 18 months).

The detailed task and collection description will be available in each task's web site.


The 4th NTCIR Workshop selected the following areas of research as "Tasks"; For details, please visit each task's call for participation below.

1. Cross-Lingual Information Retrieval Task (CLIR)
Multilingual CLIR
Bilingual CLIR and Pivot Bilingual CLIR, and
Single language IR
Languages: Traditional Chinese, Korean, English and Japanese
2. Patent Retrieval Task (PATENT)
Main task: Invalidity search
Feasibility study: a step for patent map automatic creation
3. Question Answering Task (QAC)
Task 1: 5 possible answers
Task 2: Only one set of all the answers (penalty points will be given to incorrect answers)
Task 3: A series of questions or follow-up questions
4. Text Summarization Task (TSC)
To be announced     TSC's HP
5. Web Task (WEB) (tentative CFP)
A. Informational Retrieval
B. Navigational Retrieval
C. Geographical Information
D. Topical Classification


The CLIR test collection includes Traditional Chinese, Korean, Japanese, English documents published in East Asia in 1998-1999. The size of the English and Korean document collections were increased, and the Korean document collection published in the same year range is available. NTCIR-3 CLIR, the test collections used in CLIR at NTCIR-3 is also provided as training collections.

The PATENT test collection has been extended to 10-year collection of Japanese patent, and English abstracts. NTCIR-3 PATENT, the test collection used in PATENT at NTCIR-3 is available for training as well as research purpose use.

QAC and TSC use Japanese newspaper articles, published in 1998-1999, and the number of the newspaper sources are increased, i.e., using the news articles in the newspapers published in the same years by two different companies.

WEB uses NW100G-01, the same Web document collection as those for NTCIR-3. NTCIR-3 WEB, the test collection used in Web Retrieval Task at NTCIR-3 is available for training.

For the detailed information on the data used for the 4th NTCIR Workshop, please visit NTCIR-4 DATA Home.


Please consult How to Participate. Online registration form is linked from the "How to Participate"



HOW TO PREPARE PROCEEDINGS-> Please click here for the instruction

HOW TO PREPARE WORKING NOTES-> Please click here for the instruction


2003-02-10   Call for Participation (CLIR and PATENT) Done
2003-02-21   Call for Participation (Question Answering, QAC) Done
  Release of User Agreements Forms Done
2003-03-01   Formal CFP for NTCIR-4 Done
2003-03-20   Registration Due
(registration for task participation)
2003-03-25   Deadline for task registration was extended to April 20.

*Late registration is acceptable. If you want to register after April 20, please contact Noriko Kando, the program chair of the ntcir at kando, or NTCIR secretaries at ntc-secretariat.
2003-03-30   Document Data Release Done
Each task's evaluation schedule varies, but each formal run is planned sometime during October, 2003-January, 2004.
2004-02-20   Evaluation Results Release  
2004-03-31 Deadline of Camera-ready manuscript for Working Notes of the Meeting**  
2004-06-02/04 Meeting, NII, Tokyo, Japan  
2004-07-31 NEW! Deadline of Camera-ready manuscript for the Formal Proceedings of the Workshop.  
2004-October NEW! Publication of Proceedings (Both Paper and Online)   
by 2005-03-31   Task data/test collections will be available for research purpose use  

**: The selected paper(s) will be recommended to Special Issue on NTCIR-4 in ACM Transaction on Asian Language Information Processing (ACM-TALIP)

[Japanese] [NTCIR-4]
Last modified : 2004-07-06
Modified : 2004-03-02
Modified : 2003-07-16
Modified on : 2003-03-25
This page started on : 2003-02-10