The Ninth NTCIR Workshop (NTCIR-9) Task Selection Committee has selected
the following four Core Tasks and three Pilot Tasks. For details, please
visit each taskfs homepage.
- Subtopic mining subtask: given a query,
list the different possible intents (senses or aspects).
- Ranking subtask: selectively diversify web search results.
- One Click Access (g1CLICKh) subtask: for each web search query, return an
X-byte text, not a list of URLs, in order to satisfy the user immediately.
Recognizing Inference in TExt (gRITEh)
- Binary classification subtask: Judge whether
Text1 entails Text2 or not.
- Multi-class classification subtask: Judge
whether (1) Text1 entails Text2; (2) Text2 entails Text1; (3) Text1 and
Text2 are equivalent; (4) Text1 and Text2 are contradictory; or (5) Text1
and Text2 are independent of each other.
- RITE4QA: extrinsic evaluation of binary
classification in the context of factoid question answering.
Geotemporal Information Retrieval (gGeoTimeh)
- Document retrieval for WHEN and WHERE questions
IR for Spoken Documents (gSpokenDoch)
- Spoken term detection subtask: find the
position of a given query term within spontaneous speech data.
- Spoken document retrieval subtask: retrieve
relevant passages from spontaneous speech data.
Cross-lingual Link Discovery (gCrossLinkh)
- Given an English Wikipedia page, extract
anchors from its text and provide a link to the Chinese/Korean/Japanese
Wikipedia page for each anchor.
Interactive Visual Exploration (gVis-Exh)
- Interactive complex question answering
subtask: gather as many relevant nuggets as possible within a given time
- Summarizing trend information subtask:
summarize a given time series information e.g. cabinet approval rating
over a certain period.
- Workshop: discuss and design modular evaluation
for interactive information access.
Patent Machine Translation (gPatentMTh)
- Translate Chinese/Japanese patents in to
English; and English patents into Japanese.
WHY SHOULD YOU PARTICIPATE?
- To evaluate and improve your ideas and
systems within a set time frame
- To compare different approaches and systems
from different research teams on a common test bed
- To build reliable evaluation data sets
- To build a community of researchers from
around the world to discuss their common interests and to innovate collectively
- To redefine the task and improve its evaluation
- And to have fun
WHAT PARTICIPANTS MUST DO
- Sign registration and user agreement forms
(See HOW TO PARTICIPATE)
- Follow the task guidelines/schedule and
submit your system output (grunsh) to your task organizers
- Based on your task participation experience,
write a paper for the NTCIR-9 final meeting (which will be presented in
a poster session, or a poster AND an oral presentation session)
- For the above purpose, at least one person
from your participating team must physically attend the NTCIR-9 final meeting
(Dec 6-9, 2011@NII, Tokyo)
- Report to NII every time you publish a
paper using NTCIR data
HOW TO PARTICIPATE
Please consult the 'How to Participate' page. An online registration form is available there.
NTCIR is a series of evaluation workshops for fostering research in information
access technologies – cross-lingual and multilingual ones in particular.
An NTCIR final meeting takes place at NII, Tokyo every 18 months or so
to conclude the tasks and to move on to the next challenges. The First
NTCIR (NTCIR-1) took place in 1999. See the NTCIR website for a
complete history (including online proceedings) of NTCIR.
Last updated: Februrary 16, 2011