NTCIR (NII Test Collection for IR Systems) Project bNTCIRbCONTACT INFORMATIONbNIIb
NTCIR HOME

NTCIR-9 HOME
-Workshop Aims
-NEWS
-Call for Participation
-Sponsors
-Program
-Keynote
-Registration
-Venue + Travel
Accomodation
Travel Support
VISA
-Submission Instruction
-Evaluation Tasks
INTENT
Vis-EX
RITE
CrossLink
GeoTime
PatentMT
SpokenDoc
Call for Task Proposals
Call for Participation
Data
-EVIA 2011
Call for Papers
-Organizers
Coordination
Program Committee
EVIA PC
-Important Dates
-Task Participation
-Contact Us
NTCIR HOME
Past NTCIRs
Data/Tools
Proceedings
FAQ


The 9th NTCIR Workshop Meeting

Program

bProgram at a Glanceb
b
Keynote and Report OutbLanguage Resource Exhibition (TBA)b
bDAY-1 & EVIA 2011bDAY-2bDAY-3bDAY-4 b

DAY-1 December 6 Tuesday: Breakout Sessions & EVIA 2011
9:30 - Registration Open (9:30-18:00)

10:00-12:00

Breakout Sessions
-CrossLink: 10:00-11:00, Room 2010, NII
-1CLICK: 10:00-11:00, Room 2001, NII
-GeoTime: 10:00-12:00, Room 1210, NII
-RITE: 10:00-12:00, Room 1208, NII
-INTENT: 11:00-12:00, Room 2001, NII
-PatentMT: 11:00-12:00, Room 2010, NII

12:00 - 13:00 Lunch break*
13:00 - 17:00 The 4th International Workshop on Evaluating Information Access (EVIA 2011)
Co-Chairs: Kazuaki Kishida, Mark Sanderson and William Webber
13:00 - 15:00 Presentation
Session Chair: Mark Sanderson (RMIT University, Melbourne, Australia) [Online Proceedings]
13:00 - 13:30 The Effectiveness of Cross-lingual Link Discovery
Ling-Xiang Tang (Queensland University of Technology, Australia), Kelly Y. Itakura (Queensland University of Technology, Australia), Shlomo Geva (Queensland University of Technology, Australia), Andrew Trotman (University of Otago, New Zealand), and Yue Xu (Queensland University of Technology, Australia)
13:30 - 14:00 A Micro-analysis of Topic Variation for a Geotemporal Query
Fredric Gey (University of California, Berkeley, USA), Ray R. Larson (University of California, Berkeley, USA), Jorge Machado (National Institute of Electroniques and Computer Systems(INESC-ID), Lisbon, Portugal), and Masaharu Yoshioka (Hokkaido University, Japan)
14:00 - 14:30 What Makes a Good Answer in Community Question Answering? An Analysis of Assessors' Criteria
Daisuke Ishikawa (National Institute of Informatics, Japan), Noriko Kando (National Institute of Informatics, Japan) and Tetsuya Sakai (Microsoft Research Asia, P.R.China)
14:30 - 15:00 Evaluation of Interactive Information Access System using Concept Map
Masaharu Yoshioka (Hokkaido University, Japan), Noriko Kando (National Institute of Informatics, Japan) and Yohei Seki (University of Tsukuba, Japan)
15:00 - 15:30 Cofee Break
15:30 - 16:45 Reports from Other Evaluation Campaigns
Session Chair: William Webber (The University of Maryland, USA) [Online Proceedings]
15:30 - 15:45 TREC/TAC by Ian Soboroff (National Institutet of Standards and Technology, USA)
15:45 - 16:00 MediaEval by Gareth Jones (Dublin City University, Ireland)
16:00 - 16:15 INEX by Andrew Trotman (The University of Otago, New Zealand) and Shlomo Geva (Queensland University of Technology, Australia)
16:15 - 16:30 CLEF by Nicola Ferro (The university of Padua, Italy) and Hideo Joho (The University of Tsukuba, Japan)
16:30 - 16:45 FIRE by Hideo Joho (The University of Tsukuba, Japan)
16:45 - 17:30 Panel Discussion:
TREC is 20 years old, Where Now for Evaluation Campaigns? [Session Note] [Online Proceedings]
Session Chair:
William Webber (The University of Maryland, USA)
Speakers:
Ian Soboroff (TREC/TAC)
Gareth Jones (MediaEval)
Andrew Trotman and Shlomo Geva (INEX)
Hideo Joho (NTCIR)

Mark Sanderson (RMIT University, Melbourne, Australia)
* Lunch is not provided

DAY-2 December 7 Wednesday: NTCIR-9 Meeting
8:45 - Registration Open (8:45-17:00)
9:30 - 9:45 Opening, Welcome by Noriko Kando (National Institute of Informatics, Japan)
9:45 - 10:30 Overview of NTCIR-9
Tetsuya Sakai (Microsoft Research Asia, P.R.China) and Hideo Joho (The University of Tsukuba, Japan)
[Online Proceedings]
10:30 - 11:30 Keynote
Title:Natural Language understanding, Semantic-based Information Retrieval and Knowledge Management
Jun'ichi Tsujii (Microsoft Research Asia)

[Online Proceedings]
Session Chair: Eiichiro Sumita (National Institute of Information and Communications Technology, Japan)
Ten minute break
11:40 - 13:20 Session 1: NTCIR-9 Patent Machine Translation Task (PatentMT)
Session Chair:
Isao Goto (National Institute of Information and Communications Technology, Japan)
[Online Proceedings]
11:40 - 12:05 Overview of the Patent Machine Translation Task at the NTCIR-9 Workshop
Isao Goto (National Institute of Information and Communications Technology, Japan), Bin Lu (City University of Hong Kong / Hong Kong Institute of Education, Hong Kong), Ka Po Chow (Hong Kong Institute of Education, Hong Kong), Eiichiro Sumita (National Institute of Information and Communications Technology, Japan), and Benjamin K. Tsou (Hong Kong Institute of Education / City University of Hong Kong)
12:05 - 12:20 BBN's Systems for the Chinese-English Sub-task of NTCIR-9 PatentMT Evaluation
Jeff Ma and Spyros Matsoukas (BBN Technologies, USA)
12:20 - 12:35 NTT-UT Statistical Machine Translation in NTCIR-9 PatentMT
Katsuhito Sudoh, Kevin Duh, Hajime Tsukada, Masaaki Nagata (NTT Communication Science Labs., Japan), Xianchao Wu, Takuya Matsuzaki and Jun'ichi Tsujii (The University of Tokyo, Japan)
12:35 - 12:50 The NiuTrans Machine Translation System for NTCIR-9 PatentMT
Tong Xiao, Qiang Li, Qi Lu, Hao Zhang, Haibo Ding, Shujie Yao, Xiaoming Xu, Xiaoxu Fei, Jingbo Zhu, Feiliang Ren and Huizhen Wang (Northeastern University, P.R. China)
12:50 - 13:05 The RWTH Aachen System for NTCIR-9 PatentMT
Minwei Feng, Christoph Schmidt, Joern Wuebker, Stephan Peitz, Markus Freitag and Hermann Ney (RWTH Aachen University, Germany)
13:05 - 13:20 IBM Chinese-to-English PatentMT System for NTCIR-9
Young-Suk Lee, Bing Xiang, Bing Zhao, Martin Franz, Salim Roukos and Yaser Al-Onaizan (IBM T. J. Watson Research Center, USA)

13:20 - 14:40

Poster Session 1: PatentMT (with Buffet Lunch) [Poster List]

14:40 - 16:00

Session 2: NTCIR-9 Geotemporal Information Retrieval Task (GeoTime)
Session ChairFMasaharu Yoshioka (Hokkaido University, Japan)
[Online Proceedings]
14:40 - 15:00 NTCIR9-GeoTime Overview - Evaluating Geographic and Temporal Search: Round 2
Fredric Gey (University of California, Berkeley, USA), Ray R. Larson (University of California, Berkeley, USA), Jorge Machado (National Institute of Electroniques and Computer Systems(INESC-ID), Lisbon, Portugal), and Masaharu Yoshioka (Hokkaido University, Japan)
15:00 - 15:15 NTCIR-9 GeoTime at Osaka Kyoiku University - Toward Automatic Extraction of Place/Time Terms -
Takashi Sato (Osaka Kyoiku University, Japan)
15:15 - 15:30 Geo-temporal Information Retrieval Based on Semantic Role Labeling and Rank Aggregation
Yoonjae Jeong, Gwan Jang, Kyung-Min Kim and Sung-Hyon Myaeng (Korean Advanced Institute of Science and Technology, Korea)
15:30 - 15:45 RMIT and Gunma University at NTCIR-9 GeoTime Task
Michiko Yasukawa (Gunma University, Japan), Shane Culpepper, Falk Scholer and Matthias Petri (RMIT University, Melbourne, Australia)
15:45 - 16:00 The Use of Inference Network Models in NTCIR-9 GeoTime
Christopher G. Harris (The University of Iowa, USA)

16:00 - 16:20

Cofee Break

16:20 - 17:20

Session 3: NTCIR-9 Interactive Visual Exploration (VisEx)
Session Chair: Mitsunori Matsushita (Kansai University, Japan)
[Online Proceedings]
16:20 - 16:40 Overview of the VisEx task at NTCIR-9
Tsuneaki Kato (The University of Tokyo, Japan), Mitsunori Matsushita (Kansai University, Japan) and Hideo Joho (University of Tsukuba, Japan)
16:40 - 17:00 Grid-based Interaction for NTCIR-9 VisEx Task
Hideo Joho (University of Tsukuba, Japan) and Tetsuya Sakai (Microsoft Research Asia, P.R.China)
17:00 - 17:20 Read Article Management in Document Search Process for NTCIR-9 VisEx Task
Yasufumi Takama, Shunichi Hattori and Ryosuke Miyake (Tokyo Metropolitan University, Japan)
18:00-20:00 Banquet


DAY-3 December 8 Thursday: NTCIR-9 Meeting
8:45 - Registration Open (8:45-17:00)
9:20 - 10:30 Invited Talks:
Session Chair: Tsuneaki Kato (The University of Tokyo, Japan)
[Online Proceedings]
9:20 - 9:55 Invited Talk 1
Title:Principles for Robust Evaluation Infrastructure
Mark Sanderson (RMIT University, Melbourne, Australia ) and William Webber (The University of Maryland, USA )
9:55 - 10:30 Invited Talk 2:
Title: Information Retrieval Experimentation with Terrier: Scaling Up in Size and Language Support
Iadh Ounis (University of Glasgow, UK)

10:30 - 10:50

Cofee Break

10:50 - 12:35

Session 4: NTCIR-9 Recognizing Inference in TExt Task(RITE)
Session Chair: Teruko Mitamura (Carnegie Mellon University, USA)
[Online Proceedings]
10:50 - 11:15 Overview of NTCIR-9 RITE: Recognizing Inference in TExt
Hideki Shima (Carnegie Mellon University, USA), Hiroshi Kanayama (IBM Research - Tokyo, Japan), Cheng-Wei Lee (Academia Sinica, Taiwan), Chuan-Jie Lin (National Taiwan Ocean University, Taiwan), Teruko Mitamura (Carnegie Mellon University, USA), Yusuke Miyao (National Institute of Informatics, Japan), Shuming Shi (Microsoft Research Asia, P.R.China), Koichi Takeda (IBM Research - Tokyo, Japan)
11:15 - 11:35 A Machine Learning based Textual Entailment Recognition System of JAIST Team for NTCIR9 RITE
Quang Nhat Minh Pham, Le Minh Nguyen, and Akira Shimazu (Japan Advanced Institute of Science and Technology, Japan)
11:35 - 11:55 Predicate-argument Structure based Textual Entailment Recognition System of KYOTO Team for NTCIR9 RITE
Tomohide Shibata and Sadao Kurohashi (Kyoto University, Japan)
11:55 - 12:15 UIOWA at NTCIR-9 RITE: Using the Power of the Crowd to Establish Inference Rules
Christopher G. Harris (The University of Iowa, USA)
12:15 - 12:35 ICRC_HITSZ at RITE: Leveraging Multiple Classifiers Voting for Textual Entailment Recognition
Yaoyun Zhang, Jun Xu, Chenlong Liu, Xiaolong Wang, Ruifeng Xu, Qingcai Chen, Xuan Wang, Yongshuai Hou and Buzhou Tang (Harbin Institute of Technology, P.R.China) 
12:35 - 14:00 Poster Session 2: GeoTime, VisEx and RITE (with buffet lunch)[Poster List]
14:00 - 15:20 Session 5: NTCIR-9 Cross-lingual Link Discovery Task (CrossLink)
Session Chair: Shlomo Geva (Queensland University of Technology, Australia)
[Online Proceedings]
14:00 - 14:20 Overview of the NTCIR-9 Crosslink Task: Cross-lingual Link Discovery
Ling-Xiang Tang (Queensland University of Technology, Australia), Shlomo Geva (Queensland University of Technology, Australia), Andrew Trotman (University of Otago, New Zealand), Yue Xu (Queensland University of Technology, Australia) and Kelly Y. Itakura(Queensland University of Technology, Australia)
14:20 - 14:40 HITSf Graph-based System at the NTCIR-9 Cross-lingual Link Discovery Task
Angela Fahrni Vivi Nastase and Michael Strube (Heidelberg Institute for Theoretical Studies, Germany)
14:40 - 15:00 UKP at CrossLink: Anchor Text Translation for Cross-lingual Link Discovery
Jungi Kim and Iryna Gurevych (Technische Universitat Darmstadt, Germany)
15:00 - 15:20 English-to-Korean Cross-linking of Wikipedia Articles at KSLP
In-Su Kang and Ralph Marigomen (Kyungsung University, Korea)

15:20 - 15:50

Cofee Break
15:50 - 17:10 Session 6: NTCIR-9 IR for Spoken Documents (SpokenDoc)
Session Chair: Tomoyosi Akiba (Toyohashi University of Technology, Japan)
[Online Proceedings]
15:50 - 16:10 Overview of the IR for Spoken Documents Task in NTCIR-9 Workshop
Tomoyosi Akiba (Toyohashi University of Technology, Japan), Hiromitsu Nishizaki (University of Yamanashi, Japan), Kiyoaki Aikawa (Tokyo University of Technology, Japan), Tatsuya Kawahara (Kyoto University, Japan) and Tomoko Matsui (The Institute of Statistical Mathematics, Japan)
16:10 - 16:25 Spoken Term Detection Using Multiple Speech Recognizers' Outputs at NTCIR-9 SpokenDoc STD subtask
Hiromitsu Nishizaki, Hiroto Furuya, Satoshi Natori and Yoshihiro Sekiguchi (University of Yamanashi, Japan)
16:25 - 16:40 High speed spoken term detection by combination of n-gram array of a syllable lattice and LVCSR result for NTCIR-SpokenDoc
Keisuke Iwami and Seiichi Nakagawa (Toyohashi University of Technology, Japan)
16:40 - 16:55 Spoken document retrieval method combining query expansion with continuous syllable recognition for NTCIR-SpokenDoc
Satoru Tsuge (Daido University, Japan), Hiromasa Ohashi (Nagoya University, Japan), Norihide Kitaoka (Nagoya University, Japan), Kazuya Takeda (Nagoya University, Japan) and Kenji Kita (The University of Tokushima, Japan)
16:55 - 17:10 DCU at the NTCIR-9 SpokenDoc Passage Retrieval Task
Maria Eskevich and Gareth J. F. Jones (Dublin City University, Ireland)


DAY-4 December 9 Friday
8:45 - Registration Open (8:45-15:00)
9:15 - 10:55 Session 7: NTCIR-9 INTENT Task (INTENT)
Session Chair: Min Zhang (Tsinghua University, P.R.China)
[Online Proceedings]
9:15 - 9:45 Overview of the NTCIR-9 INTENT Task
Ruihua Song (Microsoft Research Asia, P.R.China), Min Zhang (Tsinghua University, P.R.China), Tetsuya Sakai (Microsoft Research Asia, P.R.China), Makoto Kato (Kyoto University, Japan), Yiqun Liu (Tsinghua University, P.R.China), Miho Sugimoto (National Institute of Informatics, Japan), Qinglei Wang (Xi'an Jiaotong University, P.R. China) and Naoki Orii(The University of Tokyo, Japan)
9:45 - 10:02 ICTIR Subtopic Mining System at NTCIR-9 INTENT Task
Shuai Zhang, Kai Lu and Bin Wang (Institute of Computing Technology of the Chinese Academy of Sciences, P.R. China)
10:02 - 10:19 University of Glasgow at the NTCIR-9 Intent task: Experiments with Terrier on Subtopic Mining and Document Ranking
Rodrygo L.T. Santos, Craig Macdonald and Iadh Ounis (University of Glasgow, UK)
10:19 - 10:36 Microsoft Research Asia at the NTCIR-9 Intent Task
Jialong Han (Renmin University of China, P.R.China) , Qinglei Wang (Xi'an Jiaotong University, P.R. China), Naoki Orii (The University of Tokyo, Japan), Zhicheng Dou (Microsoft Research Asia, P.R.China), Tetsuya Sakai (Microsoft Research Asia, P.R.China) and Ruihua Song (Microsoft Research Asia, P.R.China)
10:36 - 10:53 THUIR at NTCIR-9 Intent Task
Yufei Xue, Fei Chen, Tong Zhu, Chao Wang, Zhichao Li, Yiqun Liu, Min Zhang, Yijiang Jin and Shaoping Ma (Tsinghua University, P.R.China)
10:55 - 11:10 Cofee Break
11:10 - 12:10 Session 8: NTCIR-9 One Click Access Subtask (1CLICK)
Session Chair: Tetsuya Sakai(Microsoft Research Asia, P.R.China)
[Online Proceedings]
11:10 - 11:25 Overview of NTCIR-9 1CLICK
Tetsuya Sakai (Microsoft Research Asia, P.R.China), Makoto Kato (Kyoto University, Japan) and Young-In Song (Microsoft Research Asia, P.R.China)
11:25 - 11:35 Information Extraction based Approach for the NTCIR-9 1CLICK Task
Makoto P. Kato, Meng Zhao, Kosetsu Tsukuda, Yoshiyuki Shoji, Takehiro Yamamoto, Hiroaki Ohshima and Katsumi Tanaka (Kyoto University, Japan)
11:35 - 11:55 TTOKU Summarization Based Systems at NTCIR-9 1CLICK task
Hajime Morita (Tokyo Institute of Technology, Japan), Takuya Makino (Tokyo Institute of Technology, Japan), Tetsuya Sakai (Microsoft Research Asia, P.R.China), Hiroya Takamura (Tokyo Institute of Technology, Japan) and Manabu Okumura (Tokyo Institute of Technology, Japan)
11:55 - 12:10 Microsoft Research Asia at the NTCIR-9 1CLICK Task
Naoki Orii (Carnegie Mellon University, USA), Young-In Song (Microsoft Research Asia, P.R.China) and Tetsuya Sakai (Microsoft Research Asia, P.R.China)
12:10 - 13:40 Poster Session 3: CrossLink, SpokenDoc and INTENT/1CLICK (with buffet lunch)[Poster List]
13:40 - 15:10 Task Proposal Discussion
15:10 - 15:40 Wrap up and QA

We are looking forward to seeing you soon!

Last Modified:2011.12.15