	DAY-1: June 7 (Tues)	DAY-2: June 8 (Wed)	DAY-3: June 9 (Thurs)		DAY-4: June 10 (Fri)
	EVIA 2016	NTCIR-12	NTCIR-12		NTCIR-12
9:00	Break-out Sessions		STC	MobileClick	MathIR	Lifelog
9:30		Welcome NTCIR-12 Overview
10:00		Welcome NTCIR-12 Overview
10:30		Keynote (Speaker: Alistair Moffat)		Coffee Break	Coffee Break
					Coffee Break	MedNLPDoc
					SpokenQuery&Doc
11:00				IMine
11:30		Coffee Break
12:00		Task Overview Temporalia, STC			Poster & Lunch Session: MathIR, SpokenQuery&Doc, Lifelog, MedNLPDoc, NTCIR-13
12:30		Task Overview Temporalia, STC	Poster & Lunch Session: QALab, MobileClick, IMine
13:00	EVIA 2016: Keynote	Poster & Lunch Session: Temporalia, STC
13:30	EVIA 2016: Keynote				Invited Talks
14:00	Paper Session1		QALab	Temporalia	Invited Talks
14:30	Coffee break	Task Overview IMine, MedNLPDoc, MobileClick, SpokenQuery&Doc			Introduction to the NTCIR-13 Tasks Wrap Up
15:00	Paper Session2
15:30
16:00			Coffee Break Break-out Sessions
16:30	Panel (The future of evaluation and EVIA)	Coffee Break
17:00	Panel (The future of evaluation and EVIA)	Task Overview: MathIR, Lifelog, QALab			Extended Steering Committee
17:30	Break-out Session
18:00			Banquet (at the Bankers Club)
18:30
19:00

9:00 -	Registration Open (9:30-17:00)
9:00 - 12:00	Breakout Sessions ・QA Lab for Entrance Exam: NII Room 1509 (10:30 - 13:00) (15th floor) ・Actionable Knowledge Graph (NTCIR-13): NII Room 2001A (11:00 - 12:00) (20th floor)

13:00 - 17:30	The 7th International Workshop on Evaluating Information Access (EVIA 2016) (at Room 2, 3, and 4) Co-Chairs: Charles Clarke (University of Waterloo, Canada), Emine Yilmaz (University College London, UK)
13:00 - 13:50	EVIA Keynote Title: Understanding and Predicting Search Satisfaction in a Heterogeneous Environment Yiqun Liu (Tsinghua University, China) Session Chair: Tetsuya Sakai (Waseda University, Japan)
13:50 - 14:30	Paper Session 1
(20 min)	An Easter Egg Hunting Approach to Test Collection Building in Dynamic Domains Seyyed Hadi Hashemi (University of Amsterdam, Netherlands), Charles Clarke and Adriel Dean-Hall (University of Waterloo, Canada), Jaap Kamps (University of Amsterdam, Netherlands)and Julia Kiseleva (Eindhoven University of Technology, Netherlands)
(20 min)	On Estimating Variances for Topic Set Size Design Tetsuya Sakai (Waseda University, Japan) and Lifeng Shang (Huawei, HongKong)
14:30 - 15:00	Coffee Break
15:00 - 16:40	Paper Session 2
(20 min)	A Laboratory-Based Method for the Evaluation of Personalised Search Camilla Sanvitto (University of Milano-Bicocca, Italy), Debasis Ganguly, Gareth J. F. Jones (Dublin City University, Ireland) and Gabriella Pasi (University of Milano-Bicocca, Italy)
(20 min)	Promoting Repeatability Through Open Runs Ellen Voorhees, Shahzad Rajput and Ian Soboroff (NIST, USA)
(20 min)	Evaluating Search Among Secrets Douglas Oard, Katie Shilton (University of Maryland, USA) and Jimmy Lin (University of Waterloo, Canada)
(20 min)	Automatic Sentence Ordering Assessment Based on Similarity Liana Ermakova (PSU, IRIT, Russian Federation)
(20 min)	Two-layered Summaries for Mobile Search: Does the Evaluation Measure Reflect User Preferences? Makoto P. Kato (Kyoto University, Japan), Virgil Pavlu (Northeastern University, USA), Tetsuya Sakai (Waseda University, Japan), Takehiro Yamamoto and Hajime Morita (Kyoto University, Japan)
16:40 - 17:30	Panel Session Future of Information Access Evaluation Panelists: Gareth J.F. Jones (Dublin City University, Ireland), Yiqun Liu (Tsinghua University, China), Douglas Oard (University of Maryland, USA), and Ian Soboroff (NIST, USA) Moderator: Charles Clarke (University of Waterloo, Canada)
17:30 - 20:00	Breakout Session ・ Short Text Conversation: NII Room 2004 & 2005 (20th floor)

8:45 -	Registration Open (8:45-17:00)
9:20 - 9:40	Welcome ( at Hitotsubashi Hall ) Noriko Kando (National Institute of Informatics Tokyo, Japan) Session Chairs: Makoto P. Kato (Kyoto University, Japan) and Kazuaki Kishida (Keio University, Japan)
9:40 - 10:20	Overview of NTCIR-12 ( at Hitotsubashi Hall ) Kazuaki Kishida (Keio University, Japan), Makoto P. Kato (Kyoto University, Japan) Session Chair: Makoto P. Kato and Kazuaki Kishida
10:20 - 11:20	Keynote ( at Hitotsubashi Hall ) Title: What Would We Like IR Metrics to Measure? Alistair Moffat, Department of Computing and Information Systems The University of Melbourne, Australia Session Chair: Mark Sanderson (RMIT University, Australia)
11:20 - 11:50	Coffee Break
11:50 - 12:50	Task Overview: Temporalia, STC ( at Hitotsubashi Hall ) Session Chairs: Makoto P. Kato and Kazuaki Kishida Temporalia-2: Overview of NTCIR-12 Temporal Information Access (Temporalia-2) Task Hideo Joho (University of Tsukuba, Japan), Adam Jatowt (Kyoto University, Japan), Roi Blanco (University of A Coruña, Spain), Haitao Yu (University of Tsukuba, Japan) and Shuhei Yamamoto (University of Tsukuba, Japan) STC: Overview of the NTCIR-12 Short Text Conversation Task Lifeng Shang (Noah's Ark Lab of Huawei, HongKong), Tetsuya Sakai (Waseda University, Japan), Zhengdong Lu (Noah's Ark Lab of Huawei, HongKong), Hang Li (Noah's Ark Lab of Huawei, HongKong), Ryuichiro Higashinaka (Nippon Telegraph and Telephone Corporation, Japan)and Yusuke Miyao (National Institute of Informatics, Japan)

14:20 - 16:20	Task Overview: IMine, MobileClick, MedNLPDoc, SpokenQuery&Doc, Lifelog, QALab, and MathIR ( at Hitotsubashi Hall ) Session Chairs: Makoto P. Kato and Kazuaki Kishida IMine-2: Overview of the NTCIR-12 IMine-2 Task Takehiro Yamamoto (Kyoto University, Japan), Yiqun Liu (Tsinghua University, China), Min Zhang (Tsinghua University, China), Zhicheng Dou (Renmin Univeristy of China, China), Ke Zhou (Yahoo!Labs, UK), Ilya Markov (University of Amsterdam, The Netherlands), Makoto P. Kato (Kyoto University, Japan), Hiroaki Ohshima (Kyoto University, Japan), Sumio Fujita (Yahoo Japan Corporation, Japan) MedNLPDoc: Overview of the NTCIR-12 MedNLPDoc Task Eiji Aramaki (Kyoto University, Japan), Mizuki Morita (Okayama University, Japan), Yoshinobu Kano (Shizuoka University, Japan) and Tomoko Ohkuma (Fuji Xerox, Japan) MobileClick-2: Overview of the NTCIR-12 MobileClick-2 Task Makoto P. Kato (Kyoto University, Japan), Tetsuya Sakai (Waseda University, Japan), Takehiro Yamamoto (Kyoto University, Japan), Virgil Pavlu (Northeastern University, USA), Hajime Morita (Kyoto University, Japan) and Sumio Fujita (Yahoo Japan Corporation, Japan) SpokenQuery&Doc-2: Overview of the NTCIR-12 SpokenQuery&Doc-2 Task Tomoyosi Akiba (Toyohashi University of Technology, Japan), Hiromitsu Nishizaki (University of Yamanashi, Japan), Hiroaki Nanjo (Kyoto University, Japan) and Gareth J. F. Jones (Dublin City University, Ireland)
16:20 - 16:50	Coffee Break
16:50 - 18:20	Task Overview: MathIR, Lifelog, QALab (at Hitotsubashi Hall) Session Chairs: Makoto P. Kato and Kazuaki Kishida MathIR: NTCIR-12 MathIR Task Overview Richard Zanibbi (Rochester Institute of Technology, USA), Akiko Aizawa (National Institute of Informatics, Japan), Michael Kohlhase (Jacobs University Bremen, Germany), Iadh Ounis (University of Glasgow, UK), Goran Topić (National Institute of Informatics, Japan) and Kenny Davila (Rochester Institute of Technology, USA) Lifelog: Overview of NTCIR-12 Lifelog Task Cathal Gurrin (Dublin City University, Ireland), Hideo Joho (University of Tsukuba, Japan), Frank Hopfgartner (University of Glasgow, UK), Liting Zhou (Dublin City University, Ireland) and Rami Albatal (Dublin City University, Ireland) QALab: Overview of the NTCIR-12 QA Lab-2 Task Hideyuki Shibuki (Yokohama National University, Japan), Kotaro Samamoto (Yokohama National University, Japan), Madoka Ishioroshi (National Institute of Informatics, Japan), Akira Fujita (National Institute of Informatics, Japan), Yoshinobu Kano (Sizuoka University, Japan), Teruko Mitamura (Carnegie Mellon University, USA), Tatsunori Mori (Yokohama National University, Japan) and Noriko Kando (National Institute of Informatics, Japan)

9:10 - 12:10	Parallel Session A (at Room1): NTCIR-12 Short Text Conversation Task (STC) Session Chairs: Lifeng Shang (Huawei, HongKong), Tetsuya Sakai (Waseda University, Japan), Ryuichiro Higashinaka (NTT Media Intelligence Laboratories, Japan)
1. (20min)	Overview of the NTCIR-12 Short Text Conversation Task Lifeng Shang (Noah's Ark Lab of Huawei, HongKong), Tetsuya Sakai (Waseda University, Japan), Zhengdong Lu (Noah's Ark Lab of Huawei, HongKong), Hang Li (Noah's Ark Lab of Huawei, HongKong), Ryuichiro Higashinaka (Nippon Telegraph and Telephone Corporation, Japan)and Yusuke Miyao (National Institute of Informatics, Japan)
2. (15min)	USTC at NTCIR-12 STC Task Junbei Zhang, Junfeng Hou, Shiliang Zhang and Lirong Dai (University of Science and Technology of China, China)
3. (15min)	The splab at the NTCIR-12 Short Text Conversation Task Ke Wu, Xuan Liu and Kai Yu (Shanghai Jiao Tong University, China)
4. (15min)	ITNLP: Pattern-based Short Text Conversation System at NTCIR-12 Yang Liu, Chengjie Sun, Lei Lin and Xiaolong Wang (Harbin Institute of Technology, China)
5. (15min)	Microsoft Research Asia at NTCIR-12 STC Task Zhongxia Chen (University of Science and Technology of China, China), Ruihua Song and Xing Xie (Microsoft Research Asia, China)
10:30 - 11:00	Coffee Break (30min)
6. (15min)	OKSAT at NTCIR-12 Short Text Conversation Task: Priority to Short Comments, Filtering by Characteristic Words and Topic Classification Takashi Sato, Yuta Morishita and Shota Shibukawa (Osaka Kyoiku University, Japan)
7. (15min)	Scoring of response based on suitability of dialogue-act and content similarity Sota Matsumoto and Masahiro Araki (Kyoto Institute of Technology, Japan)
8. (15min)	UT Dialogue System at NTCIR-12 STC Shoetsu Sato, Shonosuke Ishiwatari, Naoki Yoshinaga, Masashi Toyoda (The University of Tokyo, Japan) and Masaru Kitsuregawa (The University of Tokyo / National Institute of Informatics, Japan)
9. (15min)	Analysis of Similarity Measures between Short Text for the NTCIR-12 Short Text Conversation Task Kozo Chikai and Yuki Arase (Osaka University, Japan)
(10min)	NTCIR-12 STC Wrap-Up: Now What?

9:00 - 10:30	Parallel Session B-1 (at Room4): NTCIR-12 Mobile Information Access (MobileClick-2) Session Chair: Makoto P. Kato (Kyoto University, Japan)
1. (15min)	CUIS at the NTCIR-12 MobileClick2 Task Kwun Ping Lai and Wai Lam (The Chinese University of Hong Kong, HongKong) and Lidong Bing (Carnegie Mellon University, USA)*
2. (15min)	IRIT at the NTCIR-12 MobileClick-2 Task Abdelhamid Chellal and Mohand Boughanem (Toulouse Institute of Computer Science Research, France)*
3. (15min)	UHYG at the NTCIR-12 MobileClick Task: Link-based Ranking on iUnit-Page Bipartite Graph Sho Iizuka, Takayuki Yumoto, Manabu Nii and Naotake Kamiura (University of Hyogo, Japan)*
4. (15min)	YJST at the NTCIR-12 MobileClick-2 Task Yuya Ozawa, Taichi Yatsuka* and Sumio Fujita (Yahoo Japan Corporation, Japan)*
5. (15min)	Element-based Retrieval@MobileClick-2 Atsushi Keyaki and Jun Miyazaki (Tokyo Institute of Technology, Japan) and Kenji Hatano (Doshisha University, Japan)*
6. (15min)	RMIT at the NTCIR-12 MobileClick-2: iUnit Ranking and Summarization Subtasks Kevin Ong, Ruey-Cheng Chen and Falk Scholer (RMIT University, Australia)*

11:00 - 12:30	Parallel Session B-2 (at Room4): NTCIR-12 Search Intent and Task Mining Task (IMine-2) Session Chair: Takehiro Yamamoto (Kyoto University)
1. (18 min)	NEXTI at NTCIR-12 IMine-2 Task Hidetsugu Nanba* (Hiroshima City University, Japan), Tetsuya Sakai (Waseda University, Japan), Noriko Kando* (National Institute of Informatics, Japan), Atsushi Keyaki (Tokyo Institute of Technology, Japan), Koji Eguchi (Kobe University, Japan), Kenji Hatano (Doshisha University, Japan), Toshiyuki Shimizu (Kyoto University, Japan), Yu Hirate (Rakuten, Inc., Japan), and Atsushi Fujii (Tokyo Institute of Technology, Japan)
2. (18 min)	HUKB at NTCIR-12 IMine-2 Task: Utilization of Query Analysis Results and Wikipedia Data for Subtopic Mining Masaharu Yoshioka (Hokkaido Univeristy, Japan)*
(10 min)	Break
3. (18 min)	RUCIR at NTCIR-12 IMINE-2 Task Ming Yue, Zhicheng Dou, Sha Hu, Jinxiu Li, Xiaojie Wang, and Ji-Rong Wen (Renmin University of China, China)
4. (18 min)	THUIR at NTCIR-12 IMine Task Zeyang Liu, Ye Chen, Rongjie Cai, Jiaxin Mao, Chao Wang, Cheng Luo, Xin Li, Yiqun Liu, Min Zhang, Huanbo Luan and Shaoping Ma (Tsinghua University, China)

14:00 - 16:00	Parallel Session A-2 (at Room1): NTCIR-12 QA Lab for Entrance Exam Task (QALab) Session Chairs: Teruko Mitamura (Carnegie Mellon University, USA) and Hideyuki Shibuki (Yokohama National University, Japan)
1. (20 min)	ISOFT-Team at NTCIR-12 QA-Lab 2: Using Choice Verification Soonchoul Kwon, Seonyeong Park, Daehwan Nam, Kyusong Lee, Hwanjo Yu and Gary Ceunbae Lee (Pohang University of Science and Technology, Korea)*
2. (20 min)	NUL System at QA Lab-2 Task Mio Kobayashi, Hiroshi Miyashita, Ai Ishii and Chikara Hoshino (Nihon Unisys, Ltd., Japan)*
3. (20 min)	SML Question-Answering System for World History Essay and Multiple-choice Exams at NTCIR-12 QA Lab-2 Takuma Takada, Takuya Imagawa, Takuya Matsuzaki and Sato Satoshi (Nagoya University, Japan)*
4. (20 min)	IMTKU Question Answering System for World History Exams at NTCIR-12 QA Lab2 Min-Yuh Day*, Cheng-Chia Tsai, Wei-Chun Chuang, Jin-Kun Lin, Hsiu-Yuan Chang, Tzu-Jui Sun, Yuan-Jie Tsai, Yi-Heng Chiang, Cheng-Zhi Han, Wei-Ming Chen, Yun-Da Tsai, Yi-Jing Lin, Yue-Da Lin, Yu-Ming Guo, Ching-Yuan Chien (Tamkang University,Taiwan) and Cheng-Hung Lee (Sagacity Technology Co., Ltd., Taiwan)
5. (20 min)	KitAi-QA: A Question Answering System for NTCIR-12 QALab-2 Shuji Fukamachi and Kazutaka Shimada (Kyushu Institute of Technology, Japan)*
6. (20 min)	KSU Team's Multiple Choice QA System at the NTCIR-12 QA Lab-2 Task Tasuku Kimura, Ryosuke Nakata and Hisashi Miyamori (Kyoto Sangyo University, Japan)*

14:00 - 16:00	Parallel Session B-3 (at Room4): NTCIR-12Temporal Information Access Task (Temporalia) Session Chair: Hideo Joho (University of Tsukuba, Japan)
1. (20 min)	A Probabilistic Framework for Time-Sensitive Search: MPII at the NTCIR-12 Temporalia-2 Task Dhruv Gupta and Klaus Berberich (Max Planck Institute for Informatics, Germany)*
2. (20 min)	L3S at the NTCIR-12 Temporal Information Access (Temporalia-2) Task Zeon Trevor Fernando, Jaspreet Singh and Avishek Anand (L3S Research Center, Germany)*
3. (15 min)	WIS @ the NTCIR-12 Temporalia-2 Task Yue Zhao and Claudia Hauff (Delft University of Technology, Netherlands)*
(10min)	Break
4. (15 min)	KGO at the NTCIR-12 Temporalia Task: Exploring Temporal Information in Search Queries Xin Kang, Yunong Wu and Fuji Ren (Tokushima University, Japan)*
5. (15 min)	DUT-NLP-CH @ NTCIR-12 Temporalia TID Subtask Jiahuan Pei, Degen Huang, Jianjun Ma, Dingxin Song and Leyuan Sang (Dalian University of Technology, China)*
6. (15 min)	TUTA1 at the NTCIR-12 Temporalia Task Ning Liu, Mengjia He, Chao Li, Xin Kang and Fuji Ren (Tokushima University, Japan)*

10:45 - 12:10	Parallel Session A-2 (at Room1): NTCIR-12 Spoken Query and Spoken Document Retrieval Task (SpokenQuery&Doc) Session Chairs: Tomoyosi Akiba (Toyohashi University of Technology, Japan), Hiromitsu Nishizaki (University of Yamanashi, Japan)
(17 min)	DCU at the NTCIR-12 SpokenQuery&Doc-2 Task David N. Racca and Gareth Jones (Dublin City University, Ireland)*
(17 min)	Spoken document retrieval using neighboring documents and extended language models for query likelihood model Kazuaki Ogawa, Tatsuaki Murahashi, Hiroaki Taguchi, Koudai Nakajima, Masanori Takehara, Satoshi Tamura* and Satoru Hayamizu (Gifu University, Japan)*
(17 min)	UB at the NTCIR-12 SpokenQuery&Doc-2: Spoken Content Retrieval Using Multiple ASR Hypotheses and Syllables Jianqiang Wang (The University at Buffalo, USA)*
(17 min)	An STD system using multiple STD results and multiple rescoring method for NTCIR-12 SpokenQuery&Doc task Ryota Konno, Kazuki Ouchi, Masato Obara, Yoshino Shimizu, Takashi Chiba, Tatsuro Hirota and Yoshiaki Itoh* (Iwate Prefectural University, Japan)*
(17 min)	Combining State-level and DNN-based Acoustic Matches for Efficient Spoken Term Detection in NTCIR-12 SpokenQuery&Doc-2 Task Shuji Oishi, Tatsuya Matsuba, Mitsuaki Makino and Atsuhiko Kai (Shizuoka University, Japan)*

16:00 -	Coffee Break
(16:00) - 17:30	Break-out Sessions	Break-out Sessions Schedule & Room Guide ・ MathIR: NII Room 2006 (20th floor) ・ QA Lab for Entrance Exam: Conference Room 1 ・ Open Live Test for Question Retrieval (NTCIR-13): Conference Room 2&3 ・ Lifelog: Conference Room 4 (16:00 - 16:45) ・ Neurally Augmented Image Labelling Strategies (NTCIR-13): Conference Room 4 (16:45 - 17:30)

9:00 - 10:30	Parallel Session A-1 (at Room1): NTCIR-12 Mathematical Information Retrieval Task (MathIR) Session Chairs: Akiko Aizawa (National Institute of Informatics, Japan), Michael Kohlhase (Jacobs University Bremen, Germany) and Richard Zanibbi (Rochester Institute of Technology, USA)
(15 min)	Exploring the One-brain Barrier: A Manual Contribution to the NTCIR-12 MathIR Task Moritz Schubotz (Technische Universität Berlin, Germany), Norman Meuschke (University of Konstanz, Germany) and Marcus Leich (Technische Universität Berlin, Germany)*
(15 min)	The Math Retrieval System of ICST for NTCIR-12 MathIR Task Liangcai Gao, Ke Yuan, Yuehan Wang, Zhuoren Jiang and Zhi Tang (Peking University, China)*
(15 min)	MCAT Math Retrieval System for NTCIR-12 MathIR Task Giovanni Yoko Kristianto (The University of Tokyo, Japan), Goran Topić and Akiko Aizawa (National Institute of Informatics, Japan)*
(10min)	Break
(15 min)	Math Indexer and Searcher under the Hood: Fine-tuning Query Expansion and Unification Strategies Michal Růžička, Petr Sojka and Martin Líška (Masaryk University, Czech Republic)*
(15 min)	Tangent-3 at the NTCIR-12 MathIR Task Kenny Davila, Richard Zanibbi (Rochester Institute of Technology, USA), Andrew Kane and Frank Wm. Tompa (University of Waterloo, Canada)*
(10 min)	Discussions

13:30 - 14:30	Invited Talks (at Hitotsubashi Hall) Todai Robot Project Noriko Arai (National Institute of Informatics, Japan) News from TREC Ian Soboroff (the National Institute of Standards and Technology (NIST), USA) MediaEval 2016 Multimedia Benchmarking Initiative Gareth Jones (Dublin City University, Ireland)
14:30 - 16:00	Introduction to the NTCIR-13 Tasks Wrap Up (at Hitotsubashi Hall)

NTCIR

The 12th NTCIR Conference

DAY-1 June 7 Tuesday: Breakout Sessions & EVIA 2016

DAY-2 June 8 Wednesday: NTCIR-12 Conference (at Hitotsubashi Hall)

DAY-3 June 9 Thursday: NTCIR-12 Conference & Break-out Sessions

DAY-4 June 10 Friday: NTCIR-12 Conference