[Date Prev][Date Next][Date Index]

[ntcir:266] reminder: CFP of NTCIR-7




*************************************************************
                Call for Participation

             The 7th NTCIR Workshop (2007/2008)
Evaluation of Information Access Technologies: Information Retrieval,
       Question Answering, and Cross-Lingual Information Access

              January 2008 - December 2008
(Final Meeting: December 16-19, 2008, NII, Tokyo, Japan)

                 http://ntcir.nii.ac.jp/
*************************************************************
                    Online Registration:
http://research.nii.ac.jp/ntcir/cgi-bin/ntc7Registration.cgi?lang=en
*************************************************************

We are pleased to announce that the Seventh NTCIR (NTCIR-7)
workshop will start this year, and the concluding Workshop meeting
will be held at NII, Tokyo, Japan in December 16-19, 2008.

Participation is invited from anyone interested in research on
information access technologies and evaluation of them, such as
retrieval of documents from various genres, cross-lingual
information retrieval of Asian languages, question answering
and cross-lingual information access.

NTCIR Workshops are periodical events which are held once per
one and half years. All the documents needed for evaluation
will be provided from NII to the participants.

We have used Documents in East Asian Languages, but attracted
international participation. You are most welcome to participate!

Each task has Wiki or mailing lists for discussion. Discussion
for task design and evaluation methodologies are welcome.



** Tasks/Clusters for NTCIR-7

NTCIR-7 hosts the following tasks.

Cluster 1. Advanced Cross-lingual Information Access (ACLIA)
* Complex Cross-Lingual Question Answering (CCLQA)
* Information Retrieval for Query Answering (IR4QA)

Cluster 2. Information Access to User Generated Contents (UGC)
* Multi-Lingual Opinion Analysis Task (MOAT)
* Cross-Lingual Information Retrieval over Blog data (CLIR-B)

Cluster 3. Information Access to Focused Domains (PATENT)
* Patent Mining Task
* Patent Translation Task

Cluster Independent
* Multimodal Summarization of Trends (MuST)



** Clusters and Tasks Overview

* Cluster 1: Advanced Cross-lingual Information Access (ACLIA)
http://aclia.lti.cs.cmu.edu/wiki/moin.cgi/Home

This cluster evaluates "Complex CLQA", "CLIR" and "the contribution
of CLIR to CLQA". Documents are (Simplified and Traditional)
Chinese, and Japanese news published in 1998-2001. Questions/topics
are English, Chinese, Japanese and can be more.

CCLQA and CLIR share the same set of questions/topics, and
participation is welcome to test (1) end-to-end QA, or (2) CLIR
or IR module only using either original natural language questions,
or analyzed queries containing question types. IR modules will
be evaluated both IR itself and effectiveness in QA.

CCLQA and IR focusing to specific types of questions is new for
NTCIR. And we would like to know "What kind of IR mechanism
would be the best for what kind of QA mechanism?", "What kind
of combination is the best?", etc.



* Cluster 2: Information Access to User-Generated Contents

This cluster consists of "Cross-Lingual Information Retrieval
for Blog (CLIR-B)" task and "Multilingual Opinion Analysis
Task (MOAT)". Because of the legal re-consideration, the corpus
and tasks may be changed from what originally annouced. The
detail will be annouced later.

CLIR has been investigated in NTCIR from the beginning, but
Blog is a new document genre for NTCIR. CLIR for Blog shall be
an informational task to search opinionated documents relevant
to each topic. Opinion Analysis Task tests the ability of
the system to automatically identify relevance and opinionatedness
of each sentences in the relevant documents, opinion holder,
polarity and stakeholder (target of the opinion). Compared to
Opinion Analysis Task using news documents in NTCIR-6, identifying
"Stakeholder" is new.



* Cluster 3: Information Access for Focused Domains
http://if-lab.slis.tsukuba.ac.jp/fujii/ntcfd/index-en.html

Translation: http://if-lab.slis.tsukuba.ac.jp/fujii/ntc7patmt/index-en.html
Mining: http://www.nlp.its.hiroshima-cu.ac.jp/~nanba/ntcir-7/cfp-en.html

This cluster consists of "Patent Translation" and "Patent Mining"
tasks. It is targeting to evaluate the technologies to enhance
the information access for Patent, which were investigated
in the past NTCIRs.

Patent Translation will conduct both intrinsic and extrinsic
evaluation. Intrinsic evaluation consists of automatic
evaluation using a metrics like BLEU and human judgments.
Extrinsic evaluation adopts CLIR-task based evaluation, i.e.,
the contribution machine translation for CLIR will be tested.

Patent Mining task targets cross-genre information access
between patents and scientific papers. Abstracts of conference
papers are used as "topics" and the systems are requested
to provide appropriate International Patent Classification (IPC)
classes. It can be done as automatic categorization of paper
abstracts to IPC classes or as a cross-genre retrieval from
conference papers to patents.


* Multi-modal Summarization of Trends (MuST): TBA

Automatic identification and extraction of numeric information
related to the trends of a topic, and way of visualization will
be investigated and evaluated. For visualization, as a common
platform, an open source software for visualization will be provided.



** Important Dates

REGISTRATION DUE:  December 27, 2007
  As the user agreement forms for some of the data sets have not been
available yet, we will accept late registration after the above
mentioned due date as far as it is not affect the experiments that you
will excute for the task.

DRY RUN:    from January 2008 to April 2008
- Exact dates are specified by each task.
- The submission deadline of the Patent Translation task's dry run
  is approaching very soon: January 14, 2008.

FORMAL RUN: from 2008-02 to 2008-08
- Exact dates are specified by each task.

TASK OVERVIEW Partial Release: by September 1, 2008

EVALUATION RESULTS RETURN: by September 1, 2008

PAPERS FOR PROCEEDINGS DUE: October 1, 2008

CAMERA-READY for PROCEEDINGS DUE: November 1, 2008

FINAL MEETING: December 16-19, 2008



* Notes

1. Whether there will be a dry run or not depends on each task.
For further information, please consult the web site for each task.

2. The exact dates for the dry and formal runs are decided by each
task. For further information, please contact Noriko Kando


The registration system for NTCIR-7 task participation is online.
Please register for NTCIR-7 at ;
http://research.nii.ac.jp/ntcir/cgi-bin/ntc7Registration.cgi?lang=en


"How to Participate" to NTCIR-7:
http://ntcir.nii.ac.jp/index.php/How-to-Participate.html
    Please read through "how to paraticipate" careful and understand
what the participants are expected to do.

"User Agreement forms" for NTCIR-7 participants are:
http://ntcir.nii.ac.jp/index.php/User-Agreement.html
   To obtain the data sets used for the evaluation, each particpating
group must submit two copies of the signed user agreement forms of the
task(s) that you participate. Data will be delivered to you only AFTER
NII received the complete user agreement forms.

We are very sorry that the User agreement form for MuST has delayed, but
will be available later of this week.

For the Cluster 2 tasks (MOAT, CLIRB), currently the user agreement
forms for the training data from the past NTCIR are available.

******************************************************************