NTCIR-4 2004 Meeting: Open Submission Session
Evaluation of Information Access Technologies:
Information Retrieval, Question Answering and Summarization
June 2-4, 2004, Tokyo, Japan

The NTCIR-4 Workshop hosts an open submission session. Paper submissions are invited from anybody who are interested in the investigation of the information access technologies and the evaluation methodologies. Submissions from non-NTCIR task participants are welcome.

NTCIR is a series of evaluation workshops designed to enhance the research in information access technologies such as information retrieval, question answering, text summarization, text mining, etc., by providing infrastructure of evaluation and testing including large-scale reusable test collections, a forum of researchers, investigation of evaluation methodologies and metrics. The workshop is held once per one and half years, i.e. about 18-month interval. The NTCIR-4 selected five areas of research as "tasks"; cross-lingual information retrieval (CLIR), patent retrieval, question answering, text summarization, and Web retrieval.

NTCIR Workshop has started in late 1998 with emphasis on Japanese and English, then gradually enlarged the scope to East Asian languages, i.e. Chinese and Korea, and English documents published in East Asia, and has attracted international participation not only from Asia but also from North America, Europe, and Oceania.

This year's meeting will include a special emphasis on bringing together researchers in natural language processing, information retrieval, and other language processing technologies for information access, so that we will be able to work together towards moving this interdisciplinary research area of information access.

NTCIR-4 Workshop Meeting will held from Wednesday June 2 to Friday June 4. It includes an overview of the NTCIR-4, overview of each of the five above mentioned tasks and selected full system papers from each task, digital posters (poster with demo, or ordinary poster) by every active task participant, and full and short papers in the open submission session.

As for the open submission session, the organizers especially encourages submissions that discuss combinations of different information access technologies (ex., question answering and summarization; video, speech and information retrieval, etc.), evaluation methodologies and metrics, and any attempt to exploit the information access and evaluation on extremely large-scale unstructured dynamic document collection such as WEB.

APR 15, 2004 : Submission Due: extended abstracts or full papers
APR 30, 2004 : Notification: acceptance/rejection
MAY 25, 2004 : Camera-ready full papers, short papers, posters
JUNE 02-04, 2004 : Conference

Topics of interest include but not limited to:

Requirements: Submissions must describe original, completed, unpublished work, and include concrete evaluation results.

Multiple submission: Papers that have been or will be submitted to other meetings, publications, oral presentation as an NTCIR active task participant must report the information about multiple submission. If the paper is accepted by other conference/session/publication, the authors must advice this to the program chairs and choose which occasion the work will be presented by May 15, 2004.

Format: The submission must be electronic PDF format. For extended abstract, any format no longer than three pages in A4-sized paper including tables/figures and references. For the full paper, submissions should follow the instruction of NTCIR Working Notes guideline: two-column and no longer than eight pages including tables/figures and references.

A PDF file of the paper/abstract and a separate sheet for the information about the paper should be sent to ntc-submit as attachment of an email.

The separate sheet will include:

The submissions may be accepted for oral presentation or for presentation in a digital poster session. The accepted papers will be included in the online working notes and printed/online formal proceedings which will be published after the meeting.


The NTCIR-4 Workshop Meeting will be held at the National Center for Sciences, where the National Institute of Informatics is situated. It is central part of Tokyo and just across the street from the Royal Palace. The travel and hotel information will be available on the Workshop Web site soon.