The NTCIR Workshop is a series of evaluation workshops designed to enhance research in Japanese and Asian language text retrieval, cross-lingual information retrieval, and related text processing techniques such as summarization, extraction, etc.
The aims are;
An evaluation workshop usually provides a set of data usable for experiments and unified evaluation procedures for experiment results. Each participating group conducts research and experiments using the data provided by the NTCIR organizer with various approaches. The importance of reusable large-scale standard data collections in IR and text processing research has been widely recognized and an evaluation workshop is now recognized as a new style of active research project that facilitates research by providing the data and a forum for research idea exchange and technology transfer.
For the First NTCIR Workshop, the process was started from November, 1998, and the Workshop meeting was held on August 30 - September 1, 1999, at KKR Hotel, Tokyo. Twenty-eight groups from six countries conducted the tasks and submitted the results for the first workshop. For the Second Workshop, the process was started from June 2000 and the meeting will be held on March 7-9, 2001, NII, Tokyo and forty-six groups from eight countries have registered for the tasks.
We have looked at both traditional laboratory-typed IR system testing and evaluation of more challenging technologies. For the laboratory-typed testing, we have placed emphasis on (1) information retrieval (IR) with Japanese or other Asian languages and (2) cross-lingual information retrieval. For the challenging issues, (3) shift from document retrieval to "information" retrieval and processing, and (4) investigation for realistic evaluation, especially evaluation methods suitable for retrieval and processing of particular document-genre and its usage of the user group of the genre and so on.