タスク概要・参加者募集

第15回 NTCIR (2019 - 2020)
情報アクセス技術研究のためのテストベッドとコミュニティ
カンファレンス: 2020年12月 東京 学術総合センター


NTCIR-15 タスク参加のご案内:

情報アクセス技術向上のための協同的な取り組みに参加してみませんか?

第15回目のNTCIR、NTCIR-15では、共通のデータセットを用いて研究するタスクへの参加チームを募集中です。
情報アクセス技術の評価には、研究者の協同作業の結果として作成される「テストコレクション」に基づく評価が欠かせません。NTCIRは、数多くの研究者の協力の下で、その評価基盤の形成に過去20年以上に渡って取り組み、技術の発展に貢献してきました。そして日々開発される新しい技術に対する評価手法を模索しつつ、活動を進めております。
情報アクセス分野の学生や若手研究者のみなさん,先生方,企業で研究をなさっている方,または情報学に興味のある方々,大規模なテストコレクションを用いた検索、質問応答、自然言語処理に
関心のある研究グループは、どなたでも歓迎します。 どうぞ、奮ってご参加ください。

NTCIRについて

評価タスク

第15回NTCIR(NTCIR-15)タスク選考委員会は、以下の5つのコアタスクと2つのパイロットタスクを選定しました。
タスク紹介スライド(キックオフイベント)を下記のページからご覧いただけます:
タスク紹介スライド(キックオフイベント): http://research.nii.ac.jp/ntcir/ntcir-15/kickoff-ja.html
タスクの詳細・最新情報について、下記のタスク概要および各タスクのウェブサイトをご覧ください。

DialEval-1     FinNum-2     QA Lab-PoliInfo-2     SHINRA2020-ML     WWW-3     Data Search     MART    

コアタスク

Dialogue Evaluation ("DialEval-1")

"顧客・ヘルプデスク対話を入力とし、対話全体の品質を推定する and/or 顧客・ヘルプデスクの各ターンを分類する"

Abstract:
DQ: Given a customer-helpdesk dialogue, return an estimated distribution of dialogue quality ratings for the entire dialogue.
ND: Given a customer-helpdesk dialogue, return an estimated distribution of labels over nugget types (similar to dialogue acts) for each turn.
Data: Chinese and English
For more information, please visit: http://sakailab.com/dialeval1/

Website: http://sakailab.com/dialeval1/
Contact:

PAGE TOP


Numeral Attachment in Financial Tweets("FinNum-2")

"Numeral attachment in financial tweets"

Abstract:
In order to understanding the numeral information in depth, we proposed a task in NTCIR-14 to disambiguate the meaning of the numerals in financial social media data. However, only understanding the meanings of numerals is not enough for practical uses, because there may has more than one cashtag in a financial tweet. Understanding their semantic roles in the financial social media data is needed when mining fine-grained opinions toward a certain target.
Along this line, we design another novel task for fine-grained numeral understanding in financial social media data, called numeral attachment, which aims to detect the attached target (i.e., cashtag) of the numeral. That is, we attempt to understand that the numeral is attached to which cashtag in a tweet. For example, there are two cashtags and one numeral in (T1). The numeral "36.50" is related to $BEXP, instead of $KOG.
(T1) $KOG Took a small position- hopefully a better outcome than getting kneecapped by $BEXP selling itself dirt cheap at 36.50

Website: https://sites.google.com/nlg.csie.ntu.edu.tw/finnum2020
Contact:

PAGE TOP


Question Answering Lab for Political Information ("QA Lab-PoliInfo-2")

"地方議会会議録の要約、発話者のスタンス分類、同一条例の推定"

Abstract:
We propose the QA Lab-PoliInfo-2 (Question Answering Lab for Political Information 2) task at NTCIR 15 is aimed to verify the credibility of political information including fake news, using complex real-world question answering (QA) technologies.

Website: http://poliinfo2.net/
Contact: qalab-admin

PAGE TOP


SHINRA 2020 Multi-lingual("SHINRA2020-ML")

"30言語のWikipedia記事を拡張固有表現に分類する"

Abstract:
Based on manually categorized Japanese Wikipedia pages and the language links between Wikipedia pages in different languages, the task is to categorize the entire 30 language Wikipedia pages into Extended Named Entity. We expect the participants to use the Wikipedia pages with the link from Japanese Wikipedia as the training data, and run the system to categorize the remaining Wikipedia pages which don't have language links. After the task is over, we (including the participants) will combine the results by all the participants (i.e. by Ensemble learning), and publish the results to the public. It is a scheme called "Resource by Collaborative Contribution (RbCC)" and we are expecting many participants with a good will.

Website: http://liat-aip.sakura.ne.jp/shinra2020-ml/
Contact:

PAGE TOP


We Want Web with CENTRE ("WWW-3")

"アドホックWeb検索。複製可能性・再現可能性の検証も可"

Abstract:
The Chinese subtask is an adhoc web search task where a large query log data can be utilised. The English subtask is an adhoc web search task where not only regular adhoc runs but also replicated/reproduced runs are evaluated. All runs are required to process both the WWW-2 topics and the new WWW-3 topics.

Website: http://sakailab.com/www3/
Contact:

PAGE TOP



パイロットタスク

Data Search ("Data Search")

"統計データ検索"

Abstract:
The Data Search task focuses on the retrieval of a statistical data collection published by the Japanese government (e-Stat), and one published by the US government (Data.gov).

Website: https://ntcir.datasearch.jp/
Contact:

PAGE TOP


Micro Activity Retrieval Task ("MART")

"Detecting microactivities in a rich multi-modal lifelogging sensor streams"

Abstract:
The NTCIR15-MAD task aims to motivate the development of a first generation of techniques for high-precision micro-activity detection and retrieval of micro-activities of daily living, to support identification and retrieval of activities that occur over short time-scales, such as minutes, rather than the long-duration event segmentation tasks of the past work. Participating researchers will develop and benchmark approaches to retrieve micro-activities from rich time-aligned multi-modal sensor data. The chosen sensors will capture a lifelog camera data stream, biosignal activity (EEG, EOG, GSR, HR) and computer accesses to record interaction with digital artefacts.

Website: http://ntcir-mart.computing.dcu.ie/
Contact:

PAGE TOP



Last modified: 2019-10-17