Workshop Home
The Third NTCIR Workshop (2001/2002)
Evaluation of Information Retrieval, Q&A, and Summarization
September 2001 - October 2002
Meeting: October 8-10, 2002, NII, Tokyo Japan
enquiries: ntc-admin

The 3rd NTCIR Workshop was OVER.
For the current NTCIR Workshop (the latest in the series of NTCIR Workshops), visit  Go NTCIR Workshop !

NII National Institute of Informatics NTCIR Workshop 3
[Japanese][Home][Publication][ML][Contact Info] TASK[clir][patent][qac][tsc][web][agreement]
[Objectives] [Data] [Task] [Schedule] [Type][Application][Enquiries][Note]

An evaluation workshop of Asian language text retrieval, Q&A, and text summarization will be held from September 2001 to October, 2002. Participation is invited from anyone interested in retrieval of vairous kind of text and cross-lingual information retrieval of Asian languages from large-scale collections, and Q&A and text summarization of Japanese texts.



NTCIR-3 is over. For information on data see the NTCIR data page.

Documents will be provided for the participants those who returned required signed user agreement forms.

CLIR Task(Cross-Lingual IR) (Newspaper articles published in Asia)

- Chinese: CIRB010, United Daily News (1998-1999)
- Korean: Korea Economic Daily (1994)
- Japanese: Mainichi Newspaper (1998-1999)*
- English: Taiwan News and Chinatimes English News (1998-1999), Mainich Daily News (1998-1999)*

Patent Retrieval Task

- Japanese patents: 1998-1999 (about 17GB)
- Japio patent abstracts: 1995-1999
- Patent Abstracts of Japan (English translations for Japio patent abstracts): 1995-1999
- Patolis test collection (34 search topics for Japanese patents and relevance assessment)
- Newspaper articles (Japanese/English/Traditional Chinese)

Question Answering Task (qac)

Japanese newspaper articles (Mainichi Newspaper 1998-1999)*

Automatic Text Summarization Task (tsc2)

Japanese newspaper articles (Mainichi Newspaper 1998-1999)*

Web Retrieval Task (web)

Web documents collected from mainly .jp domain. Size: Large (ca. 100GB), Small (ca.10GB)
Available at the" Open-Lab" in the NII. Text processing such as indexing must be done on the machines in the Lab.


Below is a brief summary of the tasks envisaged for the Workshop. A participant will conduct one or more of the tasks or subtasks below. Participation in only one subtask (for example Japanese monolingual IR (J-J)) is available:

CLIR Task (Cross-Lingual IR) APPLICATION
Documents and topics will be provided in four languages.

Patent Retrieval TaskAPPLICATION

Question Answering Task (qac)APPLICATION

qac website:

Automatic Text Summarization Task (tsc2) APPLICATION

tsc website;

Web Retrieval Task (web)APPLICATION


2001-06-25 Task Plan Meeting: TSC2, QAC finished
2001-07-23 Task Plan Meeting: CLIR, Patent Search and Web Search finished
-- 2001-07-25
IPSJ SIGFI: Special Session for NTCIR (including discussion on NTCIR WS 3 tasks) finished
2001-08-01 Formal CFP for NTCIR WS 3 finished
2001-09-30 Application Due finished
2001-10-01 Document release (newspaper): delayed to 2001-10-15 finished
-- 2002-01
Dry Run and Round-Table Discussion (depends on each task) finished
2001-12 Open Lab start: Web Retrieval finished
-- 2002-03
Formal Run (depends on each task) finished
-- 2002-07
Round-Table Discussion (depends on each task) finished
2002-07-01 Evaluation Results Delivery finished
2002-08-20 Paper for Working Note Due finished
-- 2002-10-10
NTCIR Workshop 3 Meeting
Days 1-2: Closed session
Day 3: Open session
2002-12-01 Paper for Final Proceedings Due
This schedule is an outline of all tasks. Please check each task schedule on each CFP page of the tasks.
Some tasks extended the deadline for application. For details, please consult ntc-admin or each task organizers.


The list of the participating groups will be made public although the evaluation results will be announced using the group IDs only. Whichever of the types of participation, every participating group must submit (1) paper(s) for the workshop proceedings, (2) a system description form which describes your system, and (3) bibliographicreferences and a copy of all your papers when you will publish a paper using NTCIR test collections.


Online application is available at:

If you will have trouble with online registration, please download the registration form for each task, complete and return it via e-mail, fax, or postal mail to;

NTCIR Project Office
National Institute of Informatics (NII)
2-1-2 Hitotsubashi, Chiyoda-ku,Tokyo 101-8430, Japan
email: ntc-secretariat
fax: +81-3-4212-2751 phone: +81-3-4212-2750



[Objectives][Data][Task][Schedule][Type][Application][Enquiries][Note][top of this page]
[Japanese][Home][Publication][ML][Contact Info] TASK[clir][patent][Q&A][tsc][web][user agreement form]
Last modified: 2002-10-25