NTCIR-8 Meeting Program

DAY-1 June 15 Tuesday: Introduction & EVIA 2010

10:00 - Registration Open (10:00-17:30) Round-Table Meeting
(based on request) Extended Steering Committee

11:00 - 12:00 Introduction to NTCIR*
by Noriko Kando
(for new commers)

12:00 - 13:00 Lunch break**

13:00 - 18:10 The 3rd International Workshop on Evaluating Information Access (EVIA 2010)
Co-Chairs: Tetsuya Sakai, Mark Sanderson &
William Webber

13:00 - 13:50 Invited Talk
Microsoft's Bing and User Behavior Evaluation
John Nave (Search Technology Center, Microsoft)
[abstract & bio]

13:50 - 14:35 POOL DEPTH

13:50 -
14:05 Estimating Pool-depth on Per Query Basis (SHORT)
Sukomal Pal, Mandar Mitra and Samaresh Maiti (Indian Statistical Institute, India)

14:05 -
14:35 The Effect of Pooling and Evaluation Depth on Metric Stability
William Webber, Alistair Moffat and Justin Zobel (University of Melbourne, Australia)

Ten minute break

14:45 - 16:00 CLEF, NTCIR AND TREC COLLECTIONS

14:45 -
15:00 10 Years of CLEF Data in DIRECT: Where We Are and Where We Can Go (SHORT)
Maristella Agosti, Giorgio Maria Di Nunzio, Marco Dussin and Nicola Ferro (University of Padua, Italy)

15:00 -
15:30 Ranking Retrieval Systems without Relevance Assessments - Revisited
Tetsuya Sakai and Chin-Yew Lin (Microsoft Research Asia, China)

15:30 -
16:00 Test Collection Diagnosis and Treatment
Ian Soboroff (NIST, USA)

Ten minute break

16:10 - 17:10 DIVERSITY

16:10 -
16:40 Simple Evaluation Metrics for Diversified Search Results
Tetsuya Sakai (Microsoft Research Asia, China), Nick Craswell (Microsoft, USA), Ruihua Song (Microsoft Research Asia, China), Stephen Robertson (Microsoft Research Cambridge, UK), Zhicheng Dou (Microsoft Research Asia, China) and Chin-Yew Lin (Microsoft Research Asia, China)

16:40 -
17:10 Constructing a Test Collection with Multi-Intent Queries
Ruihua Song (Microsoft Research Asia, China), , Dongjie Qi (Dalian University of Technology, China), Hua Liu (Hong Kong University of Science and Technology, Hong Kong), Tetsuya Sakai (Microsoft Research Asia, China), Jian-Yun Nie (University of Montreal, Canada), Hsiao-Wuen Hon(Microsoft Research Asia, China) and Yong Yu (Shanghai Jiao Tong University, China)

17:10 - 18:10 WE'RE ONLY HUMAN

17:10 -
17:40 The Influence of Expectation and System Performance on User Satisfaction with Retrieval Systems
Katrin Lamm, Thomas Mandl, Christa Womser-Hacker and Werner Greve (University of Hildesheim, Germany)

17:40 -
18:10 A Game-based Evaluation Method for Subjective Tasks Involving Text Content Analysis
Keun Chan Park, Jihee Ryu, Kyung-min Kim and Sung Hyon Myaeng (Korea Advanced Institute of Science and Technology, Korea)

* Basically for new comers
** Lunch is not provided

DAY-2 June 16 Wednesday: NTCIR-8 Meeting

8:45 - Registration Open (8:45-17:00)

9:00 - 9:30 Light Refreshment

9:30 - 9:50 Opening, Welcome

9:50 - 10:30 Overview for NTCIR-8 by Noriko Kando

10:30 - 11:30 Invited Talk
Building Watson: A Grand Challenge in Automatic NL Question Answering
David A. Ferrucci and Koichi Takeda (IBM Research)
[abstract & bio]

Ten minute break

11:40 - 12:40 Overview of the NTCIR-8 ACLIA Tasks: Complex Cross Lingual Question Answering (CCLQA) and Information Retrieval for Question Answering (IR4QA)

11:40 - 12:10 Overview of the NTCIR-8 ACLIA Tasks:　Advanced Cross-Lingual Information Access
Teruko Mitamura (Carnegie Mellon University, USA), Hideki Shima (Carnegie Mellon University, USA), Tetsuya Sakai (Microsoft Research Asia, China), Noriko Kando (National Institute of Informatics, Japan), Tatsunori Mori (Yokohama National University, Japan), Koichi Takeda (IBM Research - Tokyo, Japan), Chin-Yew Lin (Microsoft Research Asia, China), Ruihua Song (Microsoft Research Asia, China), Chuan-Jie Lin (National Taiwan Ocean University, Taiwan) and Cheng-Wei Lee (Academia Sinica, Taiwan)

12:10 - 12:40 Overview of NTCIR-8 ACLIA IR4QA
Tetsuya Sakai (Microsoft Research Asia, China), Hideki Shima (Carnegie Mellon University, USA), Noriko Kando (National Institute of Informatics, Japan), Ruihua Song (Microsoft Research Asia, China), Chuan-Jie Lin (National Taiwan Ocean University, Taiwan), Teruko Mitamura (Carnegie Mellon University, USA), Miho Sugimoto (National Institute of Informatics, Japan) and Cheng-Wei Lee (Academia Sinica, Taiwan)

12:40 - 14:00 Poster Session 1 (with Buffet Lunch) [Poster List]

14:00 - 15:30 Session 1: : NTCIR-8 Advanced Cross-Lingual Information Access (ACLIA) - Complex Cross Lingual Question Answering (CCLQA)
Session Chair: Teruko Mitamura

14:00 -
14:05 Introduction to NTCIR-8 ACLIA Tasks by Teruko Mitamura

14:05 -
14:30 An Open-domain Question Answering System for NTCIR-8 C-C Task
Guiping Zhang, Wubei Zhang, Yu Bai, Shengqiao Kang and Peiyan Wang (Shenyang Institute of Aeronautical Engineering, China)

14:30 -
14:55 WHU Question Answering System at NTCIR-8 ACLIA Task
Han Ren, Donghong Ji and Jing Wan (Wuhan University, China)

14:55 -
15:20 Bootstrap　Pattern Learning for Open-Domain CLQA
Hideki Shima and Teruko Mitamura (Carnegie Mellon University, USA)

15:20 -
15:30 Opne Discussions

Coffee break

15:50 - 17:10 Session 2: : NTCIR-8 Advanced Cross-Lingual Information Access (ACLIA) - Information Retrieval for Question Answering (IR4QA)
Session Chair: Tetsuya Sakai

15:50 -
15:55 Introduction to NTCIR-8 IR4QA by Tetsuya Sakai

15:55 -
16:15 IMU Experiment in IR4QA at NTCIR-8
Xiangdong Su, Xueliang Yan, Guanglai Gao and Hongxi Wei (Inner Mongolia University, China)

16:15 -
16:35 Query Expansion from Wikipedia and Topic Web Crawler on CLIR
Meng Chun Lin, Ming-Xiang Li, Chih-Chuan Hsu and Shih-Hung Wu (Chaoyang University of Technology, Taiwan)

16:35 -
16:55 KECIR: An Information Retrieval System for IR4QA Task
Dongfeng Cai, Shengqiao Kang, Yu Bai and Peiyan Wang (Shenyang Institute of Aeronautical Engineering, Taiwan)

16:55-
17:10 Open Discussions

17:10 - 17:50 Proposal for New Tasks for NTCIR-9

Ten minute break

18:00 - 19:00 Breakout Session

DAY-3 June 17 Thursday: NTCIR-8 Meeting

8:45 - Registration Open (8:45-17:00)

9:00 - Light Refreshment

9:15 - 10:00 Report Out from other evaluations (From CLEF, TREC, and Rakuten) (Invited)

9:15 - 9:35 CLEF, CLEF 2010, and PROMISEs: Perspectives for the Cross-Language Evaluation Forum
Nicola Ferro (the University of Padua, Italy)
[abstract & bio]

9:35 - 9:55 ClueWeb09 and TREC Diversity
Charles Clarke (University of Waterloo, Canada)
[abstract & bio]

9:55 - 10:00 E-Commerce Data through Rakuten Data Challenge
Masahiro Sanjo (Rakuten Institute of Technology) and
Satoshi Sekine (Rakuten Institute of Technology, New York/ New York University)
[abstract & bio]

10:00 - 12:00 Session 3: : NTCIR-8 Geotemporal Information Retrieval (GeoTime)
Session Co-chairs: Fredric Gey and Ray Larson

10:00 -
10:25 NTCIR-GeoTime Overview: Evaluating Geographic and Temporal Search
Fredric Gey (University of California, Berkeley, USA), Ray Larson (University of California, Berkeley, USA), Noriko Kando (National Institute of Informatics, Japan), Jorge Machado (National Institute of Electroniques and
Computer Systems(INESC-ID), Lisbon, Portugal), Tetsuya Sakai (Microsoft Research Asia, China)

10:25 -
10:45 On a Combination of Probabilistic and Boolean IR Models for GeoTime Task
Masaharu Yoshioka (Hokkaido Univeristy, Japan)

10:45 -
11:05 Experiments with Geo-Temporal Expressions filtering and query expansion at document and phrase context resolution
Jorge Machado, Jose Borbinha and Bruno Martins (National Institute of Electroniques and Computer Systems(INESC-ID), Lisbon, Portugal

11:05 -
11:25 A Method for GeoTime Information Retrieval Based on Question Decomposition and Question Answering
Tatsunori Mori (Yokohama National University, Japan)

11:25 -
11:35 Experiments with Semantic-flavored Query Reformulation of Geo-Temporal Queries
Nuno Cardoso and Mario J. Silva (University of Lisbon, Portugal)

11:35 -
11:45 Vocabulary-based Re-ranking for Geographic and Temporal Searching at NTCIR GeoTime Task
Kazuaki Kishida (Keio University, Japan)

11:45 -
12:00 Open Discussions

12:00 - 13:30 Poster Session 2 (with buffet lunch)[Poster List]

13:30 - 15:30 Session 4: : NTCIR-8 PATENT Mining (PATMN)
Session Chair: Hidetsugu Nanba

13:30 -
14:00 Overview of the Patent Mining Task at the NTCIR-8 Workshop
Hidetsugu Nanba (Hiroshima City University, Japan), Atsushi Fujii (Tokyo Institute of Technology, Japan), Makoto Iwayama (Hitachi, Ltd. / Tokyo Institute of Technology) and Taiichi Hashimoto (Advanced Industrial Science and Technology)

14:00 -
14:22 Multiple Strategies for NTCIR-8 Patent Mining at BCMI
Gang Jin, Qi Kong, Jian Zhang, Xiaolin Wang, Cong Hui, Hai Zhao, and Bao-Liang Lu (Shanghai JiaoTong University, China)

14:22 -
14:44 Automatic IPC encoding and novelty tracking for effective patent mining
Douglas Teodoro, Emilie Pasche, Dina Vishnyakova (BiTeM Group, Medical Informatics Service, University of Geneva, Switzerland), Julien Gobeill, Patrick Ruch (BiTeM Group, Library and Information Sciences Department, University of Applied Sciences, Switzerland) and Christian Lovis (BiTeM Group, Medical Informatics Service, University of Geneva, Switzerland)

14:44 -
15:06 Feature-Rich Information Extraction for the Technical Trend-Map Creation
Risa Nishiyama, Yuta Tsuboi, Yuya Unno and Hironori Takeuchi (IBM Research - Tokyo, Japan)

15:06 -
15:28 Extracting Technology and Effect Entities in Patents and Research Papers
Jingjing Wang, Han Tong Loh and Wen Feng Lu (National University of Singapore, Singapore)

15:30 - 16:30 Coffee break and Breakout Session

16:30 - 17:30 Session 5: : NTCIR-8 Community QA (CQA)
Session Chair: Tetsuya Sakai

16:30 -
17:00 Overview of the NTCIR-8 Community QA Pilot Task (PartI, PartII)
Daisuke Ishikawa (National Institute of Informatics, Japan), Tetsuya Sakai (Microsft Research Asia, China) and Noriko Kando (National Institute of Informatics, Japan)

17:00 -
17:15 Microsoft Research Asia with Redmond at the NTCIR-8 Community QA Pilot Task
Young-In Song, Jing Liu, Tetsuya Sakai, Xin-Jing Wang, Guwen Feng, Yunbo Cao (Microsft Research Asia, China), Hisami Suzuki (Microsoft Research Redmond, USA) and Chin-Yew Lin (Microsft Research Asia, China)

17:15 -
17:30 ASURA: A Best-Answer Estimation System for NTCIR-8 CQA Pilot Task
Daisuke Ishikawa (National Institute of Informatics, Japan)

18:30-20:30 Banquet

DAY-4 June 18 Friday

8:45 - Registration Open (8:45-17:00)

9:00 - 9:30 Light Refreshment

9:30 - 11:30 Session 6: : NTCIR-8 PATENT Translation (PATMT)
Session Chair: Atsushi Fujii

9:30 -
10:05 Overview of the Patent Translation Task at the NTCIR-8 Workshop
Atsushi Fujii (Tokyo Institute of Technology, Japan), Masao Utiyama (National Institute of Information and Communications Technology, Japan), Mikio Yamamoto (University of Tsukuba, Japan),
Takehito Utsuro (University of Tsukuba, Japan), Terumasa Ehara (Yamanashi Eiwa College, Japan), Hiroshi Echizen-ya (Hokkai-Gakuen University, Japan) and Sayori Shimohata (Oki Electric Industry CO., Ltd., Japan)

10:05 -
10:30 MaTrEx: the DCU MT System for NTCIR-8
Tsuyoshi Okita, Jie Jiang, Rejwanul Haque, Hala Al-Maghout, Jinhua Du, Sudip Naskar and Andy Way (Dublin City University, Ireland)

10:30 -
10:55 Machine Translation for Patent documents combining rule-based translation and statistical post-editing
Terumasa Ehara (Yamanashi Eiwa College, Japan)

10:55 -
11:20 System Description of NiCT SMT for NTCIR-8
Keiji Yasuda, Taro Watanabe, Masao Utiyama and Eiichiro Sumita (National Institute of Information and Communications Technology, Japan)

11:30 - 13:00 Poster Session 3 (with buffet lunch)[Poster List]

13:00 - 15:00 Session 7: : NTCIR-8 Multilingual Opinion Analysis (MOAT)
Session Chair: Yohei Seki

13:00 -
13:40 Overview of Multilingual Opinion Analysis Task at NTCIR-8 - A Step Toward Cross Lingual Opinion Analysis -
Yohei Seki (Toyohashi University of Technology, Japan), Lun-Wei Ku (National Taiwan University, Taiwan), Le Sun (Institute of Software, Chinese Academy of Sciences, China), Hsin-Hsi Chen (National Taiwan University, Taiwan) and Noriko Kando (National Institute of Informatics, Japan)

13:40 -
14:00 Opinion Detection by Combining Machine Learning & Linguistic Tools
Olena Zubaryeva and Jacques Savoy (University of Neuchatel, Switzerland)

14:00 -
14:20 PKUTM Experiments in NTCIR-8 MOAT Task
Chenfeng Wang, Tengfei Ma, Liqiang Guo, Xiaojun Wan and Jianwu Yang (Institute of Computer Science and Technology of Peking University, China)

14:20 -
14:40 Supervised Approaches and Dependency Parsing for Chinese Opinion Analysis at NTCIR-8
Bin Lu (City University of Hong Kong, Hong Kong), Benjamin K. Tsou (City University of Hong Kong, Hong Kong) and Tao Jiang (ChiLin Star Corp, Southern Software Park, Zhuhai, China)

14:40 -
15:00 The OpAL System at NTCIR 8 MOAT
Alexandra Balahur, Ester Boldrini, Andres Montoyo and Patricio Martinez-Barco (University of Alicante, Department of Software and Computing Systems, Spain)

15: 00 - 15:30 Wrap-up

16:30 - 18:30 Extended Steering Committee

If you would like to have a round-table meeting or workshop for tasks done at NTCIR-8 or tasks proposed for NTCIR-9, please let us know. We will arrange the meeting places. The following time slots are available with pre-booking.

TIME SLOTs available for Round-table/workshop for task(s).

Monday, June 14: all day
Tuesday, June 15: 9:00-13:00
Wednesday, June 16: after 19:00-
Friday, June 18: after 15:30-
Saturday, June 19: all day

The meeting place will be available first-come-first-served basis. Earlier arragnement will be appreciative.

We are looking forward to seeing you soon!

Last updated: June 14, 2010