[ntcir:194] CFP for NTCIR-6

[Noriko Kando <kando@xxxxxxxxx>]

**********************************************************
         Call for Participation to NTCIR-6
Evaluation of Information Access Technologies: IR, QA and
     Cross-lingual Information Access Technologies

　http://research.nii.ac.jp/ntcir/ntcir-ws6/
      http://research.nii.ac.jp/ntcir/
Final Meeting: 15-18 May 2007, NII, Tokyo, Japan
**********************************************************


The NTCIR is a series of evaluation workshop to enhance the
research in information access technologies, including text
retrieval, question answering, cross-lingual information
access,　etc, by providing infrastructure of evaluation
and research including test collections, evaluation metrics
and methodologies, and a forum of researchers.

NTCIR has used East Asian Language documents but attracted
international participation. For example the previous NTCIR
has participants from 15 different countries and areas including
Australia, Canada, China PRC, Germany, Hong Kong, Ireland, Japan,
Korea, Netherlands, Singapore, Spain, Switzerland, Taiwan ROC,
UK, United States.


TASKS:

NTCIR-6 selected the following 4 areas of research as
"Tasks" and 1 area as a "pilot workshop"; Other "pilot tasks"
can be started any time during the process of NTCIR-6.


1. Cross-Lingual Information Retrieval Task (CLIR)
    Multi- and Bi-lingual CLIR, and
    Single language IR
    Languages: Traditional Chinese, Korean, and Japanese.
    Simplified Chinese can be added. To conclude the CLIR to
    news documents, 4 CLIR test collections, NTCIR-3 through -6
    will be used and Cross-collection analysis will be done.
    New metrics for graded relevance judgments are used.
    Discussion about evaluation metrics and methodologies
    are also welcome.

2. Cross-Language Question Answering Task (CLQA)
    Focus on Named Entities, which are one of the problems
    in CL information access in Asian context.
    Traditional Chinese, English and Japanese. Korean language
    is under consideration. Volunteer for cooperation to
    organize Korean part is welcome.

3. Patent Retrieval Task (PATENT)
    Retrieval task: "Invalidity search" Using Japanese patents
    (10 year patent application fulltexts) and US patents.
    Use 4 patent collections of NTCIR-3 through -6.
    Classification task: Multi-viewpoint categorization. The
    purpose is to categorize target patent applications based
    on the F-term classification system.

4. Question Answering Task (QAC)
    Question answering beyond factoid questions and their evaluation
    method. Focusing to complicated questions like "WHY".

5. Pilot Tasks
    Any attempts to test the problems to be solve in short
    time, or feasibility studies for the future tasks. To be
    announced later. Opinion Extraction, Multilingual Multi-
    document Summarization, Evaluation of WEB Search engine
    were now under consideration, but not decided

6. Pilot Workshop: Multimodal Summarization of Trend Information (MuST)
　  Focusing numerical information in the text. Extract numerical
information representing the trends from multiple documents,
analyze, summarize and display visullay. We constructed an annotated
corpus and share it. Research groups from various disciplinaries share
the corpus and work various direction of research using the corpus.
Currently using Japanese documents only. To be announced later.



WHAT'S NEW to NTCIR-6:

(1) Cross-Collection Evaluation:

In IR related tasks, multiple test collections for the same
document genres and the same users' tasks will be used and
analyze the variability across the collections.


(2) Submission Raw Data and Evaluation Results:

All the submitted runs and their evaluated results using the metrics
set by the task will be available for the active participants of the
task. The purpose of this is that we invited all the task participants
to discuss, analyze or examine the evaluation metrics and evaluation
results of the task.

Evaluation is very critical issue for all of the researchers. So
please examine how the evaluation is done and how the metrics behave,
or whether there are any methods to overcome the limitation of
current practice of the evaluation. With your cooperation, we would
like to obtain fruitful examination of the evaluation results and metrics.


OPEN SUBMISSION SESSION:

Conjuction with NTCIR-6 Final Meeting, the separate conference called
"Open Submission Session" will be held. This is a refereed international
conference on research in Information Access and their evaluation.
Detailed information will be announced later.



IMPORTANT DATES:

    Registration Deadline:             2006-05-31
    Document Set Release:              2006-06-01
    Dry Run:                      from 2006-07 to 2006-09
    Formal Run：                  from 2006-10 to 2006-12
    Evaluation Results Return:      by 2007-02-01
    Paper for the Proceedings Due :    2007-03-01
    Final Meeting, Tokyo, Japan:       2007-05-15/18



** ONLINE REGISTRATION IS AVAILABLE NOW !!


INQUIRY:
Noriko Kando at kando (at) nii.ac.jp


Your participation is more than welcome. Please join and work together!


Noriko.
---
Noriko Kando
ntcir project