Overview

NTCIR Workshop

go to NTCIR Project Website

The NTCIR Workshop is a series of evaluation workshops designed to enhance research in Japanese and Asian language text retrieval, cross-lingual information retrieval, and related text processing techniques such as summarization, extraction, etc.

The aims are;

  1. to encourage research in IR and related areas by providing large-scale data collections reusable for experiments and a common evaluation infrastructure allowing cross-system comparisons
  2. to provide a forum for research groups interested in cross-system comparison and exchanging research ideas in an informal atmosphere
  3. to investigate evaluation methods of IR and text processing techniques and methods for constructing a large-scale data set reusable for experiments.

An evaluation workshop usually provides a set of data usable for experiments and unified evaluation procedures for experiment results. Each participating group conducts research and experiments using the data provided by the NTCIR organizer with various approaches. The importance of reusable large-scale standard data collections in IR and text processing research has been widely recognized and an evaluation workshop is now recognized as a new style of active research project that facilitates research by providing the data and a forum for research idea exchange and technology transfer.

For the First NTCIR Workshop, the process was started from November, 1998, and the Workshop meeting was held on August 30 - September 1, 1999, at KKR Hotel, Tokyo. Twenty-eight groups from six countries conducted the tasks and submitted the results for the first workshop. For the Second Workshop, the process was started from June 2000 and the meeting will be held on March 7-9, 2001, NII, Tokyo and forty-six groups from eight countries have registered for the tasks.

We have looked at both traditional laboratory-typed IR system testing and evaluation of more challenging technologies. For the laboratory-typed testing, we have placed emphasis on (1) information retrieval (IR) with Japanese or other Asian languages and (2) cross-lingual information retrieval. For the challenging issues, (3) shift from document retrieval to "information" retrieval and processing, and (4) investigation for realistic evaluation, especially evaluation methods suitable for retrieval and processing of particular document-genre and its usage of the user group of the genre and so on.