WORKSHOP OBJECTIVES
To encourage research in information retrieval, cross-lingual
information retrieval and text summarization by providing
reusable test collections.
To provide a forum for research groups interested in comparing
results and exchanging ideas or opinions in an informal atmosphere.
To improve the quality of the test collections based on the
feedback from participants.
CHINESE IR TASKS (Chinese and English-Chinese IR)
Detailed information added!
The training
set and the testing set of Chinese Text Retrieval Tasks are selected from the Chinese
Information Retrieval Benchmark 1 (CIRB-1).
The CIRB-1 consists of three parts: 1)
Document Set; 2) Topic Set; and 3) Relevance Judgment.
Now, the Document Set
contains 132,173 news articles from 5 news agencies in Taiwan, the Topic Set
contains 50 topics in a form of user's information need from briefs to details, and the
Relevance Judgment consists of the related documents to the various topics.
The details of Chinese IR task could be referred to the
http://lips.lis.ntu.edu.tw/cirb/events-1.htm.
The related information is also in the same web site:
http://lips.lis.ntu.edu.tw/cirb/index.htm.
JAPANESE & ENGLISH IR TASKS (Japanese, English and English-Japanese IR)
Training set: NTCIR-1 CD,
more than 330,000
author abstracts of conference papers; more than half are Japanese-English
paired (document alignments); alignments are known and usable for
training;
Test set: NTCIR-1 and NTCIR-2.
Segmented Japanese texts are available
for both Japanese documents and topics NTCIR-1 & 2;
use of this data in NOT mandatory.
TEXT SUMMARIZATION TASK
We use newpaper articles (Mainichi Shinbun, The Mainichi:
you need to pay for a license to use) as original texts for
these tasks. They are not limited to business domain,
and articles of other domains such as editorials,
columns will be included.
Application was CLOSED
August 10, 2000: NTCIR-2 CD (new documents and fifty topics) will be distributed to the participants of Japanese & English IR tasks.
August 31, 2000: CIRB-1-CH CD (132,172 documents and 50 Chinese topics)
will be distributed to the participants of Chinese IR Task, and CIRB-1-EN CD
(132,172 documents and 50 English topics) will be distributed to the participants
of English-Chinese IR Task. (Chinese IR tasks)
September 18, 2000: Search results submission (Japanese & English IR)
October 8, 2000: Dryrun (Text Summarization)
October 20, 2000: Search results submission (Chinese IR)
November, 2000: Evaluation (Text Summarization)
January 10, 2001: Results of Relevance Assessments will be distributed to the participants (Chinese IR, Japanese IR)
February 12, 2001: Papers for the working-note proceedings submission (All Tasks)
March 7-9, 2001: Workshop meeting at NII, Tokyo, Japan.
Day 1: Open to public, Days 2-3: Active participants only
March 16, 2001: Camera-ready copies for the proceedings
Chinese Information Retrieval Task:
The Chinese IR Task is to assess the capability of participating systems
in retrieving Chinese documents using Chinese queries,
The English-Chinese IR Task is to assess the
capability of participating systems in retrieving
Chinese documents using English queries, Chinese texts,
which are composed of characters without explicit word boundary,
make the retrieval task more challengeable than English ones.
The participating systems can employ any approaches. Either
word-based or character-based systems are acceptable.
The organizer will not provide any segmentation tools
and Chinese dictionaries.
Japanese & English Information Retrieval Task:
Japanese and/or English monolingual IR; cross-lingual IR of single
language documents and mixed-language documents of English and Japanese
by Japanese and/or English topics; to investigate the search
effectiveness of systems that search a static set of documents
Text Summarization Task: automatic text summarization of Japanese texts; the aims are (1) to collect
quarified text data for summarization in Japanese. We will have
newspaper articles summarized by hand, and make them available
for research purpose use, (2) to evaluate text summarization systems;
an extrinsic evaluation, task based evaluation. For details please visit
http://galaga.jaist.ac.jp:8000/tsc/index-en.html
A. FULL: Submit retrieval results and describe the system.
The correspondence between the group name and the group ID will be
also announced.
B. ANONYMOUS: Submit retrieval results.
The details of the system may not be reported.
The correspondence between the group name and the group ID is not announced.
This category is mainly for the participants from the companies who have
troubles to report the details.
The list of the participating groups is made public but the evaluation results will be announced using the group IDs only. Whichever of the types of participation, every participating group must submit (1)a paper for the workshop proceedings, (2)a system description form which describes your system, and (3)bibliographic references and a copy of all your papers using NTCIR test collections.
Please send email to Noriko Kando, project manager, at
kando
,
or to NTCIR Project administrators (ntc-admin).
About "Chinese IR Task", please send email to the Task Chairs,
Hsin-Hsi Chen 
)
or Kuang-Hua Chen (khchen
).
About "Text Summarization Task", please send email to the Task
Chairs, Manabu Okumura
)
or Takahiro Fukushima
(fukusima
).
The first day of the Workshop meeting will be open forum of the
researchers who are interested in the topics. The second and third
days will be open only to the active participating groups that have
submited results and selected people from organizing agencies
The proceedings will be published online as well as printed-form.
Dissemination of the research results using the NTCIR collections
other than in the Workshop's Proceedings is welcome. However, the
conditions of participation preclude specific advertising claims based
on the results using the Collection or the Workshop.
International participants are welcome. Announcements will be in
English and Japanese, and English for Chinese IR Task.
The official language for the proceedings papers and presentation at the
Workshop meeting in February, 2001 is English.
For the Japanese & English IR task, because of the copyright issue, the participants
are appointed as collaborative researchers in the NII's Collaborative
Research Program. There are no additional terms regarding the
program except results submission and a paper for the proceedings.
Information about freely available Japanese morphological analysers
will be provided.
An evaluation of Korean text retrieval is organized separately by Prof Sung Hyon Myaeng, Korea
), We keep close relationship between each other,.