NTCIR

The 15th NTCIR (2019 - 2020)
Evaluation of Information Access Technologies
Conference: December 08-11, 2020, NII, Tokyo, Japan

Call for Participation to the NTCIR-15 Tasks:

Task Participation

NTCIR Aims: http://research.nii.ac.jp/ntcir/ntcir-15/aims.html

The fourteenth NTCIR (NTCIR-15) Task Selection Committee has selected the following five Core Tasks and two Pilot Tasks.
For task slides of Kick-Off event, please visit:
Slides for task introduce at the NTCIR-15 Kick-Off Event: http://research.nii.ac.jp/ntcir/ntcir-15/kickoff.html
For details and latest information, please see below and visit each task’s homepage.

DialEval-1     FinNum-2     QA Lab-PoliInfo-2     SHINRA2020-ML     WWW-3     Data Search     MART    

CORE TASKS

Dialogue Evaluation ("DialEval-1")

"Given a customer-helpdesk dialogue, estimate the quality of the entire dialogue and/or classify each customer/helpdesk turn."

Abstract:
DQ: Given a customer-helpdesk dialogue, return an estimated distribution of dialogue quality ratings for the entire dialogue.
ND: Given a customer-helpdesk dialogue, return an estimated distribution of labels over nugget types (similar to dialogue acts) for each turn.
Data: Chinese and English
For more information, please visit: http://sakailab.com/dialeval1/

Website: http://sakailab.com/dialeval1/
Contact:

PAGE TOP

Numeral Attachment in Financial Tweets("FinNum-2")

"Numeral attachment in financial tweets"

Abstract:
In order to understanding the numeral information in depth, we proposed a task in NTCIR-14 to disambiguate the meaning of the numerals in financial social media data. However, only understanding the meanings of numerals is not enough for practical uses, because there may has more than one cashtag in a financial tweet. Understanding their semantic roles in the financial social media data is needed when mining fine-grained opinions toward a certain target.
Along this line, we design another novel task for fine-grained numeral understanding in financial social media data, called numeral attachment, which aims to detect the attached target (i.e., cashtag) of the numeral. That is, we attempt to understand that the numeral is attached to which cashtag in a tweet. For example, there are two cashtags and one numeral in (T1). The numeral "36.50" is related to $BEXP, instead of $KOG.
(T1) $KOG Took a small position- hopefully a better outcome than getting kneecapped by $BEXP selling itself dirt cheap at 36.50

Website: https://sites.google.com/nlg.csie.ntu.edu.tw/finnum2020
Contact:

PAGE TOP

Question Answering Lab for Political Information ("QA Lab-PoliInfo-2")

"Generating a summary of speeches in Japanese local assembly minutes, classifying a political stance of a speaker, and identifying a mention in a speech as the same real world entity"

Abstract:
We propose the QA Lab-PoliInfo-2 (Question Answering Lab for Political Information 2) task at NTCIR 15 is aimed to verify the credibility of political information including fake news, using complex real-world question answering (QA) technologies.

Website: http://poliinfo2.net/
Contact: qalab-admin

PAGE TOP

SHINRA 2020 Multi-lingual("SHINRA2020-ML")

"Categorizing 30 language Wikipedia into Extended Named Entity categories"

Abstract:
Based on manually categorized Japanese Wikipedia pages and the language links between Wikipedia pages in different languages, the task is to categorize the entire 30 language Wikipedia pages into Extended Named Entity. We expect the participants to use the Wikipedia pages with the link from Japanese Wikipedia as the training data, and run the system to categorize the remaining Wikipedia pages which don't have language links. After the task is over, we (including the participants) will combine the results by all the participants (i.e. by Ensemble learning), and publish the results to the public. It is a scheme called "Resource by Collaborative Contribution (RbCC)" and we are expecting many participants with a good will.

Website: http://shinra-project.info/shinra2020ml/?lang=en
Contact:

PAGE TOP

We Want Web with CENTRE ("WWW-3")

"Adhoc web search, with optional replicability and reproducibility runs"

Abstract:
The Chinese subtask is an adhoc web search task where a large query log data can be utilised. The English subtask is an adhoc web search task where not only regular adhoc runs but also replicated/reproduced runs are evaluated. All runs are required to process both the WWW-2 topics and the new WWW-3 topics.

Website: http://sakailab.com/www3/
Contact:

PAGE TOP

PILOT TASK

Data Search ("Data Search")

"Ad-hoc retrieval for statistical data"

Abstract:
The Data Search task focuses on the retrieval of a statistical data collection published by the Japanese government (e-Stat), and one published by the US government (Data.gov).

Website: https://ntcir.datasearch.jp/
Contact:

PAGE TOP

Micro Activity Retrieval Task ("MART")

"Detecting microactivities in a rich multi-modal lifelogging sensor streams"

Abstract:
The NTCIR15-MAD task aims to motivate the development of a first generation of techniques for high-precision micro-activity detection and retrieval of micro-activities of daily living, to support identification and retrieval of activities that occur over short time-scales, such as minutes, rather than the long-duration event segmentation tasks of the past work. Participating researchers will develop and benchmark approaches to retrieve micro-activities from rich time-aligned multi-modal sensor data. The chosen sensors will capture a lifelog camera data stream, biosignal activity (EEG, EOG, GSR, HR) and computer accesses to record interaction with digital artefacts.

Website: http://ntcir-mart.computing.dcu.ie/
Contact:

PAGE TOP