The 13th NTCIR Conference

Program

DAY-1: Dec 5 (Tue) DAY-2: Dec 6 (Wed) DAY-3: Dec 7 (Thu) DAY-4: Dec 8 (Fri)
EVIA 2017 NTCIR-13 NTCIR-13 NTCIR-13
9:00 STC-2 AKG WWW NAILS
9:10
9:20
9:30 Welcome & NTCIR-13 Overview
9:40
9:50
10:00 Keynote
(Speaker: Omar Alonso)
10:10
10:20
10:30 Coffee Break Coffee Break
10:40
10:50 OpenLiveQ ECA
11:00 Coffee Break STC-2 MedWeb
11:10
11:20
11:30 Task Overview: Lifelog-2, MedWeb
11:40
11:50
12:00
12:10
12:20 Poster & Lunch
(QALab-3, Lifelog-2, WWW, NAILS, OpenLiveQ, ECA, NTCIR-14)
12:30 Lunch (on your own) Break
12:40
12:50
13:00 EVIA 2017 Poster & Lunch
(STC, AKG, MedWeb)
13:10
13:20
13:30
13:40
13:50 Reports from TREC and CLEF, Introduction to NTCIR-14, and Wrap Up
14:00 Task Overview: OpenLiveQ, QALab-3, AKG, ECA
14:10
14:20
14:30 Break
14:40
14:50 QALab-3 Break-out Session (OpenLiveQ and AKG)
15:00
15:10
15:20
15:30
15:40
15:50 Coffee Break
16:00 Coffee Break Break-out Session (STC)
16:10 QALab-3 Lifelog-2
16:20
16:30 Task Overview: NAILS, STC-2, WWW
16:40
16:50
17:00 Break-out Session (CENTRE)
17:10 Break-out Session (QALab-3)
17:20
17:30
17:40
17:50
18:00 NTCIR 20th Anniversary Session
(Speakers: Fredric C. Gey and Douglas W. Oard)
Extended Steering Committee (invited only)
18:10
18:20
18:30 Banquet
18:40
18:50

DAY-1 Dec 5 Tuesday: EVIA 2017



10:00 -Registration Open (10:00 - 17:00)

13:00 - 17:10The 8th International Workshop on Evaluating Information Access (EVIA 2017)

Co-chairs: Nicola Ferro (University of Padua) and Ian Soboroff(NIST)
13:00 - 13:10Opening
13:10 - 13:50EVIA Keynote
Title:Cross-Language Information Retrieval in the MATERIAL Program
Douglas W. Oard (University of Maryland)
13:50 - 15:00Session 1: Evaluation Measures and Test Collections
(25 min)Test Collections and Measures for Evaluating Customer-Helpdesk
Zhaohao Zeng, Cheng Luo, Lifeng Shang, Hang Li, Tetsuya Sakai
(25 min)An interval-like scale property for IR evaluation measures
Marco Ferrante, Nicola Ferro, Silvia Pontarollo
(20 min)Evaluating Evaluation Measures with Worst-Case Confidence Interval Widths
Tetsuya Sakai
15:00 - 15:30Coffee Break
15:30 - 17:00Session 2: Assessment and Automatic Evaluation
(20 min)Automatic Evaluation of World History Essay Using Chronological and Geographical Measures
Kotaro Sakamoto, Hideyuki Shibuki, Madoka Ishioroshi, Akira Fujita, Yoshinobu Kano,Teruko Mitamura, Tatsunori Mori, Noriko Kando
(25 min)Towards Automatic Evaluation of Multi-Turn Dialogues: A Task Design that Leverages Inherently Subjective Annotations
Tetsuya Sakai
(25 min)The Effect of Inter-Assessor Disagreement on IR System Evaluation: A Case Study with Lancers and Students
Tetsuya Sakai
(20 min)Unanimity-Aware Gain for Highly Subjective Assessments
Tetsuya Sakai
17:00 - 17:10Wrap-up and closing

* Lunch is not provided

PAGE TOP

DAY-2 Dec 6 Wednesday: NTCIR-13 Conference (at Hitotsubashi Hall)

8:45 -Registration Open (8:45 - 17:00)
9:30 - 9:45Welcome (at Hitotsubashi Hall )
Noriko Kando (National Institute of Informatics)
Session Chairs: Makoto P. Kato (Kyoto University) and Yiqun Liu (Tsinghua University)
9:45 - 10:00Overview of NTCIR-13 (at Hitotsubashi Hall )
Makoto P. Kato (Kyoto University) and Yiqun Liu (Tsinghua University)
Session Chairs: Makoto P. Kato (Kyoto University) and Yiqun Liu (Tsinghua University)
10:00 - 11:00Keynote (at Hitotsubashi Hall )
Title: The practice of crowdsourcing: things to know about using humans and machines for labeling
Omar Alonso (Microsoft)

Session Chair: (TBA)
11:00 - 11:30 Coffee Break
11:30 - 12:30 Task Overview: Lifelog-2 and MedWeb (at Hitotsubashi Hall )
Session Chairs: Makoto P. Kato (Kyoto University) and Yiqun Liu (Tsinghua University)

11:30 - 12:00
Lifelog-2: Overview of the NTCIR-13 Lifelog-2 Task

12:00 - 12:30
MedWeb: Overview of the NTCIR-13: MedWeb Task

12:30 - 14:00Lunch (on your own)

14:00 - 16:00Task Overview: OpenLiveQ, QALab-3, AKG, and ECA (at Hitotsubashi Hall )
Session Chairs: Makoto P. Kato (Kyoto University) and Yiqun Liu (Tsinghua University)

14:00 - 14:30
OpenLiveQ: Overview of the NTCIR-13 OpenLiveQ Task

14:30 - 15:00
QALab-3: Overview of the NTCIR-13 QA Lab-3 Task

15:00 - 15:30
AKG: Overview of NTCIR-13 Actionable Knowledge Graph (AKG) Task

15:30 - 16:00
ECA: Overview of NTCIR-13 ECA Task
16:00 - 16:30 Coffee Break
16:30 - 18:00Task Overview: NAILS, STC-2, and WWW (at Hitotsubashi Hall)
Session Chairs: Makoto P. Kato (Kyoto University) and Yiqun Liu (Tsinghua University)

16:30 - 17:00
NAILS: Overview of NTCIR-13 NAILS Task

17:00 - 17:30
STC-2: Overview of the NTCIR-13 Short Text Conversation Task

17:30 - 18:00
WWW: Overview of the NTCIR-13 We Want Web Task
18:00 - 19:00 NTCIR 20th Anniversary Session (at Hitotsubashi Hall )

Title: NTCIR from the Beginning: A Personal Research Journey
Fredric C. Gey (University of California, Berkeley)


Title: NTCIR in the World: Two Decades of Impact
Douglas W. Oard (University of Maryland)

Session Chair: (TBA)

* Lunch is not provided

PAGE TOP

DAY-3 Dec 7 Thursday: NTCIR-13 Conference

8:45 - Registration Open (8:45 - 17:00)


9:00 - 12:30, Session A-1: NTCIR-13 Short Text Conversation Task (STC-2)

9:00 - 9:18Lifeng Shang, Tetsuya Sakai, Hang Li. Overview of the NTCIR-13 Short Text Conversation Task (Chinese Subtask)
9:18 - 9:36Shih-Hung Wu, Wen-Feng Shih, Che-Cheng Yu, Liang-Pu Chen and Ping-Che Yang. CYUT-III Short Text Conversation System at NTCIR-13 STC-2 Task
9:36 - 9:54Yihan Li, Shanshan Jiang, Lei Ding, Yixuan Tong and Bin Dong. SRCB at the NTCIR-13 STC-2 Task
9:54 - 10:12Han Ni, Liansheng Lin and Ge Xu. Nders at NTCIR-13 Short Text Conversation 2 Task
10:12 - 10:30Dejian Yang, Yu Wu, Zhoujun Li, Wei Wu and Can Xu. Beihang at the NTCIR-13 STC-2 Task
10:30 - 11:00 Coffee Break (30min)
11:00 - 11:15Yunong Wu, Xin Kang, Kenji Kita and Fuji Ren. TUA1 at NTCIR-13 Short Text Conversation 2 Task
11:15 - 11:30Haizhou Zhao, Yi Du, Hangyu Li, Qiao Qian, Hao Zhou, Minlie Huang and Jingfang Xu. SG01 at NTCIR-13 STC2
11:30 - 11:45Hiroshi Nakatani, Shigenori Nishiumi, Takahiro Maeda and Masahiro Araki. KIT Dialogue System for NTCIR-13 STC-2 Task
11:45 - 12:00Hiroki Tanioka. Response Generation for Grounding at NTCIR-13 STC-2
12:00 - 12:15Toru Shimizu. YJTI at the NTCIR-13 STC Japanese Subtask

PAGE TOP



9:00 - 10:30, Session B-1: NTCIR-13 Actionable Knowledge Graph Task (AKG)

9:00 - 9:10Roi Blanco, Hideo Joho, Adam Jatowt, Haitao Yu, and Shuhei Yamamoto. Overview of NTCIR-13 AKG Task
9:10 - 9:30Xin Kang, Yunong Wu and Fuji Ren. TUA1 at the NTCIR-13 Action Mining Task: Sampling Related Actions from Online Searching
9:30 - 9:50Xinshi Lin, Wai Lam and Shubham Sharma. CUIS Team for NTCIR-13 AKG Task
9:50 - 10:10Md Mostafizur Rahman and Atsuhiro Takasu. TLAB at NTCIR-13 AKG Task
10:10 - 10:30Discussion

10:30 - 11:00 Coffee Break (30min)

11:00 - 13:00, Session B-2: NTCIR-13 Medical Natural Language Processing for Web Document (MedWeb)

11:00 - 11:15 Shoko Wakamiya, Mizuki Morita, Yoshinobu Kano, Tomoko Ohkuma and Eiji Aramaki. Overview of the NTCIR-13: MedWeb Task
11:15 - 11:30 Hayate Iso, Camille Ruiz, Taichi Murayama, Katsuya Taguchi, Ryo Takeuchi, Hideya Yamamoto, Shoko Wakamiya and Eiji Aramaki. NTCIR13 MedWeb Task: multi-label classification of tweets using an ensemble of neural networks
11:30 - 11:45 Reine Asakawa and Tomoyoshi Akiba. AKBL at the NTCIR-13 MedWeb Task
11:45 - 12:00 Anh Hang Nga Tran, Hiroko Kobayashi, Yu Sawai and Paulo Quaresma. UE and Nikon at the NTCIR-13 MedWeb Task
12:00 - 12:15 Masahito Sakishita and Yoshinobu Kano. Classification of Tweet Posters who have Diseases by Mixing Method of Rule-Based and SVM in NTCIR-13 MedWeb
12:15 - 12:25 Break
12:25 - 12:30 Chao Li, Kang Xin and Fuji Ren. Medweb Task: Identify Multi-Symptoms from Tweets Based on Active Learning and Semantic Information
12:30 - 12:35 Wei-San Lin, Hong-Jie Dai and Yu-Hsuan Shao. Principle base Approach for Classifying Tweets with Flu-related Information in NTCIR-13 MedWebTask
12:35 - 12:40 Masao Ito. NIL: Using scoring to analyse the ambiguous messages on the NTCIR-13 MedWeb task
12:40 - 12:45 Kazui Morita and Takagi Toshihisa. DrG at NTCIR­-13: MedWeb Task
12:45 - 12:50 Mamoru Sakai and Hiroki Tanioka. Keyword-based Challenges at the NTCIR-13 MedWeb
12:50 - 13:00 Discussion and Closing

PAGE TOP



13:00 - 14:30 Poster & Lunch
STC, AKG, and MedWeb [Poster List]


14:50 - 17:10, Session A-3: NTCIR-13 QA Lab for Entrance Exam Task (QALab-3)

14:50 - 15:00Tao-Hsing Chang, Yu-Sheng Tsai, Chih-Li Tsai and Pei-Xuan Cai. ATA2: A Question Answering System at NTCIR-13 QALab-3
15:00 - 15:10Kou Sato and Tetsuya Sakai. SLQAL at the NTCIR-13 QA Lab-3 Task
15:10 - 15:20Yuanzhi Ke and Masafumi Hagiwara. HagiwaraLab at the NTCIR-13 QALab-3 Task
15:20 - 15:30Kotaro Sakamoto, Madoka Ishioroshi, Yuta Fukuhara, Akihiro Iizuka, Hideyuki Shibuki, Tatsunori Mori and Noriko Kando. Forst: Question Answering System for Second-stage Examinations at NTCIR-13 QA Lab-3 Task
15:30 - 15:40Yusuke Doi, Takuma Takada, Takuya Matsuzaki and Satoshi Sato. SML Question-Answering System for World History Essay Exams at NTCIR-13 QALab-3
15:40 - 15:50Tasuku Kimura, Ryo Tagami and Hisashi Miyamori. KSU Team's QA System for World History Exams at the NTCIR-13 QA Lab-3 Task
15:50 - 16:10 Coffee Break (30min)
16:10 - 16:20Mike Tian-Jian Jiang. DGLab Question Answering System and Automatic Evaluation Method at NTCIR-13 QA Lab-3 for University Entrance Exam on World History Essay
16:20 - 16:30Min-Yuh Day, Chao-Yu Chen, Wan-Chu Huang, I-Hsuan Huang and Shi-Ya Zheng, Tz-Rung Chen, Min-Chun Kuo, Yue-Da Lin, Yi-Jing Lin. IMTKU Question Answering System for World History Exams at NTCIR-13 QA Lab-3
16:30 - 16:40Fadi Botros, Francesco Ciannella, Takaaki Matsumoto, Evan Chan, Cheng-Ta Chung, Lucas Bengtson, Keyang Xu, Tian Tian and Teruko Mitamura. CMUQA in QALab-3: Essay Question Answering for University Entrance Exams
16:40 - 16:50Takaaki Matsumoto and Teruko Mitamura. MTMT in QALab-3: World History Essay Question Answering System that Utilizes Textbooks and Open Knowledge Bases
16:50 - 17:00Tsunenori Ishioka, Kohei Yamaguchi and Tsuneori Mine. Rubric-based Automated Japanese Short-answer Scoring and Support System Applied to QALab-3


16:10 - 18:10, Session B-3: NTCIR-13 Personal Lifelog Organisation & Retrieval Task (Lifelog-2)

16:10 - 16:30Review of the Lifelog-2 Task (Cathal Gurrin)
16:30 - 16:50LSAT Sub-Task Review
- VCI2R at the LSAT Sub-Task (Quanli Xu)
- NTT at the LSAT Sub-Task (Yoshiaki Takimoto)
- DCU at the LSAT Sub-Task (Cathal Gurrin)
16:50 - 17:00LEST Sub-Task Review
- NTT at the LEST Sub-Task (Yoshiaki Takimoto)
17:00 - 17:30LIT Sub-Task Review
- Tsinghua University at the LIT Sub-Task (TBD)
- VCI2R at the LIT Sub-Task (Quanli Xu)
- DCU at the LIT Sub-Task (Cathal Gurrin)
17:30 - 17:40LAT Sub-Task Review
- NTT at the LAT Sub-Task (Yoshiaki Takimoto)
17:40 - 18:10Lifelog-3 Planning - Hideo Joho, Cathal Gurrin


Break-out Sessions
14:50 - 15:50 OpenLiveQ (at NII 20F 2005)
AKG (at Conference Room 2)
17:10 - 18:10 QALab-3 (at Conference Room 1)

18:30 - 20:30 Banquet

DAY-4 Dec 8 Friday: NTCIR-13 Conference

8:45 - Registration Open (8:45 - 15:00)


9:00 - 10:30, Session A-4: NTCIR-13 We Want Web Task (WWW)

9:00 - 9:10Cheng Luo, Tetsuya Sakai, Yiqun Liu, Zhicheng Dou, Chenyan Xiong and Jingfang Xu. Overview of the NTCIR-13 We Want Web Task
9:10 - 9:20Luke Gallagher, Joel Mackenzie, Rodger Benham, Ruey-Cheng Chen, Falk Scholer and J. Shane Culpepper. RMIT at the NTCIR-13 We Want Web Task
9:20 - 9:30Ming Yue and Zhicheng Dou. RUCIR at NTCIR-13 WWW Task
9:30 - 9:40Yukun Zheng, Cheng Luo, Weixuan Wu, Jia Chen, Yiqun Liu, Huanbo Luan, Min Zhang and Shaoping Ma. THUIR at NTCIR-13 WWW Task
9:40 - 9:50Zhuyun Dai, Chenyan Xiong and Jamie Callan. An Evaluation of the Kernel Based Neural Ranking Model in NTCIR-13 WWW
9:50 - 10:00Peng Xiao, Lingtao Li, Yimeng Fan and Tetsuya Sakai. SLWWW at the NTCIR-13 WWW Task
10:00 - 10:25Discussion about future plans: Task Organizers and Participants


9:00 - 10:30, Session B-4: NTCIR-13 Neurally Augmented Image Labelling Strategies Task (NAILS)

9:00 - 9:10Graham Healy. Overview of NTCIR-13 NAILS Task
9:10 - 9:25Amelia Solon, Stephen Gordon, Brent Lance and Vernon Lawhern. Deep Learning Approaches for P300 Classification in Image Triage: Applications to the NAILS Task
9:25 - 9:40Holly Hutson, Shlomo Geva and Philipp Cimiano. Ensemble Methods for the NTCIR-13 NAILS Task


10:30 - 10:50 Coffee Break (20min)


10:50 - 12:20, Session A-5: NTCIR-13 Open Live Test for Question Retrieval (OpenLiveQ)

10:50 - 11:00Overview of the NTCIR-13 OpenLiveQ Task
11:00 - 11:15Tomohiro Manabe, Akiomi Nishida and Sumio Fujita. YJRS at the NTCIR-13 OpenLiveQ Task
11:15 - 11:30Ming Chen, Lin Li, Yueqing Sun and Jie Zhang. Erler at the NTCIR-13 OpenLiveQ Task
11:30 - 11:45Ryo Kashimura and Tetsuya Sakai. SLOLQ at the NTCIR-13 OpenLiveQ Task
11:45 - 12:00Mengjia He, Xin Kang and Fuji Ren. TUA1 at the NTCIR-13 OpenLiveQ Task
12:00 - 12:15Takashi Sato. OKSAT at NTCIR-13 OpenLiveQ Task
12:15 - 12:20Wrap up


10:50 - 12:20, Session B-5: NTCIR-13 Emotion Cause Analysis Task (ECA)

11:00 - 11:15Qinghong Gao, Hu Jiannan, Xu Ruifeng, Gui Lin, Yulan He, Kam-Fai Wong and Qin Lu. Overview of NTCIR-13
11:15 - 11:25Xiangju Li, Shi Feng, Daling Wang and Yifei Zhang. Decision Tree Method for the NTCIR-13 ECA Task
11:25 - 11:35Han Ren, Yafeng Ren and Jing Wan. The GDUFS System in NTCIR-13 ECA Task
11:35 - 11:45Maofu Liu, Linxu Xia, Zhenlian Zhang and Yang Fu. WUST CRF-Based System at NTCIR-13 ECA Task
11:45 - 11:55What is the next step of ECA?


12:20 - 13:50 Poster & Lunch
QALab-3, Lifelog-2, WWW, NAILS, OpenLiveQ, ECA, and NTCIR-14 [Poster List]


13:50 - 14:20 Reports from TREC and CLEF
New Tracks for TREC 2018
Ian Soboroff (National Institute of Standards and Technology, NIST)

CLEF 2018: Evaluation Labs, Conference, and Initiative
Nicola Ferro (University of Padua)
14:20 - 14:55 Introduction to NTCIR-14
Session Chairs: Makoto P. Kato (Kyoto University) and Yiqun Liu (Tsinghua University)

- Personal Lifelog Organisation & Retrieval (Lifelog-3)
- Open Live Test for Question Retrieval (OpenLiveQ-2)
- QA Lab for Politics Minutes (QA Lab-4)
- Short Text Conversation (STC-3)
- We Want Web (WWW-2)
- CLEF/NTCIR/TREC REproducibility (CENTRE)
14:55 - 15:00 Wrap Up
Makoto P. Kato (Kyoto University) and Yiqun Liu (Tsinghua University)


Break-out Sessions
16:00 - 17:00 STC (at NII 20F 2005)
17:00 - 18:00 CENTRE (at NII 20F 2005)


18:00 - 20:00 Extended Steering Committee (invited only) (at NII 12F 1208 and 1210)


We are looking forward to seeing you soon!

Last modified: 2017-12-04