[Date Prev][Date Next][Date Index]

[ntcir:74] CFP ntcir ws3

apology for duplicated post.

                           CALL FOR PARTICIPATION
                    The Third NTCIR Workshop (2001/2002)
        Evaluation of Information Retrieval, Q&A, and Summarization
                       September 2001 - October 2002

               Meeting: October 8-10, 2002, NII, Tokyo Japan
               URL: http://research.nii.ac.jp/ntcir/workshop/
                        enquiries: ntcadm@xxxxxxxxx

An evaluation workshop of Asian language text retrieval, Q&A, and text
summarization will be held from September 2001 to October, 2002.
Participation is invited from anyone interested in retrieval of various
kind of text and cross-lingual information retrieval of Asian languages
from large-scale collections, and Q&A and text summarization of Japanese

   This year we picked five areas of research as task, Cross Language
Retrieval, Patent Retrieval, Question Answering, Automatic Text
Summarization, and Web Retrieval. An optional task is available in
Patent Retrieval and Web Retrieval Tasks. Any proposal using the data
provided are welcome for the optional task and we hope it will provide
an exploratory occasion for new tasks.

   * To encourage research in information retrieval, Q&A, and text
     summarization by providing reusable test collections.
   * To provide a forum for research groups interested in comparing
     results and exchanging ideas or opinions in an informal atmosphere
   * To improve the quality of the test collections based on the
     feedback from participants.

   Below is a brief summary of the tasks envisaged for the Workshop.
A participant will conduct one or more of the tasks or subtasks below.
Participation in only one subtask (for example Japanese monolingual IR
(J-J) in the CLIR Task) is available:

1. Cross Language Retrieval Task (clir)
Documents and topics are in four languages (Chinese, Korean, Japanese
and English)
   * Multilingual CLIR (MLIR): Search document collection more than one
     languages by one of four languages of topics.excepting Korean
   * Bilingual CLIR (BLIR): Search of any two different languages as
     language and documents, excepting search of English documents
   * Single Lanugage IR (SLIR): Monolingual Search of Chinese, Korea, or

DOCUMENT: newspapers publish in Asia:
- Chinese: CIRB010, United Daily News (1998-1999)
- Korean: Korea Economic Daily (1994)
- Japanese: Mainichi Newspaper (1998-1999)*
- English: Taiwan News and China English News (1998-1999),
 Mainichi Daily News (1998-1999)*

2. Patent Retrieval Task (patent)
   * Main Task
        o Cross-language Cross-DB retrieval: retrieve patents in
          response to J/E/C newspaper articles associated with
          technology and commercial products.
        o Monolingual Associative Retrieval: retrieve patents associated

          with an input Japanese patent
   * Optional task: Any research reports are invited on patent
     processing using the above data, including, but not limited to:
     generating patent maps, paraphrasing claims, aligning claims
     and examples, summarization for patents, clustering patents.
DOCUMENT: - Japanese patents: 1998-1999 (about 17GB)
- Japio patent abstracts: 1995-1999
- Patent Abstracts of Japan (English translations for
 Japio patent abstracts): 1995-1999
- Patolis test collection (34 topics and relevance assessment)
- Newspaper articles (Japanese/English/Traditional Chinese)

3. Question Answering Task (qac)
   * Task 1: System extracts five answers from the documents in some
     order. 100 questions. System is required to return support
     information for each answer of the questions. We assume
     the support information as a paragraph, 100 letter passage or
     document which includes the answer.
   * Task 2: System extracts only one answer from the documents. 100
     questions. Support information is required.
   * Task 3: evaluation of a series of questions. The related questions
     are given for the 30 of questions of Task 2.
DOCUMENT: Japanese newspaper articles (Mainichi Newspaper 1998-1999)*

4. Automatic Text Summarization Task (tsc2)
   * Task A (single document summarization): Given the texts to be
     summarized and summarization lengthes, the participants submit
     summaries for each text in plain text format.
   * Task B (multi-document summarization): Given a set of texts, the
     participants produce summaries of it in plain text format. The
     information which was used to produce the document set, such as
     queries, as well as summarization lengthes are given to the
DOCUMENT: Japanese newspaper articles (Mainichi Newspaper 1998-1999)*

5. Web Retrieval Task
   * A. Survey Retrieval (both recall and precision are evaluated)
        o A1. Topic Retrieval
        o A2. Similarity Retrieval
   * B. Target Retrieval (precision-oriented)
   * C. Optional Task
        o C1.Search Results Classification
        o C2. Speech-Driven Retrieval
        o C3. other
DOCUMENT: Web documents mainly collected from jp domain (ca.100GB &
          ca.10GB) Available at the "Open-Lab" in the NII

2001-09-30      Application Due
2001-10-01      Document release (newspaper)
2001-10/2002-01 Dry Run and Round-Table Discussion
         (depends on each task)
2001-12         Open Lab start
2001-12/2002-03 Formal Run (depends on each task)
2002-07-01      Evaluation Results Delivery
2002-08-20      Paper for Working Note Due
2002-10-08/10   NCIR Workshop 3 Meeting
             Days 1-2: Closed session (task participants only)
             Day 3: Open session
2002-12-01      Paper for Final Proceedings Due

   * A. FULL: Submit results and describe the system. The correspondence

     between the group name and the group ID will be announced.
   * B. ANONYMOUS: Submit results. The details of the system may not be
     reported. The correspondence between the group name and the group
     ID is not announced. This category is mainly for the participants
     from the companies who have troubles to report the details.

The list of the participating groups will be made public although the
evaluation results will be announced using the group IDs only. Whichever

of the types of participation, every participating group must submit
(1) paper(s) for the workshop proceedings, (2) a system description
form which describes your system, and (3) bibliographic references and
a copy of all your papers when you will publish a paper using NTCIR
test collections.

Online application;

   * Please send email to Noriko Kando, program chair or to
     NTCIR Project administrators (ntcadm@xxxxxxxxx).
   * For the details of a specific task, please contact each task's
     chair and organizers.

   * The proceedings will be published online as well as printed-form.
   * Dissemination of the research results using the NTCIR collections
     other than in the Workshop's Proceedings is welcome. However, the
     conditions of participation preclude specific advertising claims
     based on the results using the Collection or the Workshop.
   * International participants are welcome. Announcements will be in
     English and Japanese.
   * The official language for the proceedings papers and presentation
     at the Workshop meeting in October, 2002 is English.
   * Documents will be provided for the participants those who returned
   required user agreement forms.
   * DOCUMENT USAGE: The period of permitted use of Mainichi Newspapers
     and Mainichi Daily News are from 2001-09-01 to 2003-09-30. For
     active participants who submit the results and who affiliated at
     the  organization outside Japan will be able to extend the period
     up to 2008-09-30. After the permitted period will be terminated,
     the participants will have to delete all the document data. Those
     who want to use the data after the period can purchase the data
     Mainichi Newspaper Co., and obtain the permission for research
     purpose use from the company. The permitted period may vary
     according to each task.

noriko kando
ntcir project