NTCIR Workshop3:
Proceedings of the Third NTCIR Workshop on research in
information Retrieval, Automatic Text Summarization and
Question Answering (September 2001-October 2002)

- an online version of the formal proceedings -

Proceedings for other NTCIR workshops can be found at the NTCIR proceedings page.

[NTCIR Home] [NII Home]

Published
by	National Institute of Informatics (NII) 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430 Japan Phone: +81-3-4212-2750 Fax: +81-3-3556-1916
Printed in	Tokyo, Japan
by	Nihon Printing Co.,Ltd. 6-3-3 Soto-kanda, Chiyoda-ku, Tokyo 101-0021, Japan Phone: +81-3-3833-6971
Edited by	Keizo Oyama and Emi Ishida and Noriko Kando (NII)

Preface

Message from the Organizing Chair: Jun Adachi; (National Institute of Informatics)
Message from the Program Chair: Noriko Kando; (National Institute of Informatics)

1 Overview

1.1 NTCIR Workshop 3

Overview of the Third NTCIR Workshop: Noriko Kando; (National Institute of Informatics)

1.2 Cross-Lingual Information Retrieval Task

Overview of CLIR Task at the Third NTCIR Workshop: Kuang-hua Chen (*1), Hsin-Hsi Chen (*2), Noriko Kando (*3), Kazuko Kuriyama (*4), Sukhoon Lee (*5),; Sung Hyon Myaeng (*6), Kazuaki Kishida (*7), Koji Eguchi (*3), and Hyeon Kim (*8); (*1 Department of Library and Information Science, National Taiwan University, *2 Department of Computer Science and Information Engineering, National Taiwan University, *3 National Institute of Informatics, *4 School of Literature, Shirayuri College, *5 Department of Statistics, Chungnam National University, *6 Department of Computer Science, Chungnam National University, *7 Faculty of Cultural Information Resources, Surugadai University, *8 Department of Information Technology, Korea Institute of Science and Technology Information)
Characteristics of the Korean Test Collection for CLIR in NTCIR-3: Sukhoon Lee (*1), Sung Hyon Myaeng (*2), Hyeon Kim (*3), Jerry H.Seo (*3), Buil Lee (*1), and Sukhyun Cho (*2); ( *1 Department of Statistics, Chungnam National University, *2 Department of Computer Science, Chungnam National University, *3 Department. of Information Technology, Korea Institute of Science and Technology Information)

1.3 Patent Retrieval Task

Overview of Patent Retrieval Task at NTCIR-3: Makoto Iwayama (*1), Atsushi Fujii (*2), Noriko Kando (*3), and Akihiko Takano (*3); (*1 Tokyo Institute of Technology / Hitachi Ltd., *2 University of Library and Information Science / Japan Science and Technology Corporation, *3 National Institute of Informatics)

1.4 Question Answering (QAC-1) Task

Question Answering Challenge (QAC-1): An Evaluation of Question Answering Task atNTCIRWorkshop 3: Jun'ichi FUKUMOTO (*1), Tsuneaki KATO (*2), and Fumito MASUI (*3); (*1 Ritsumeikan University, *2 University of Tokyo, *3 Mie University)

1.5 Text Summarization Task

Text Summarization Challenge 2: Text Summarization Evaluation at NTCIR Workshop3: Takahiro Fukusima (*1), Manabu Okumura (*2), and Hidetsugu Nanba (*2); (*1 Otemon Gakuin University, *2 Tokyo Institute of Technology)

1.6 Web Retrieval Task

Overview of the Web Retrieval Task at the Third NTCIR Workshop: Koji Eguchi (*1), Keizo Oyama (*1), Emi Ishida (*1), Noriko Kando (*1), and Kazuko Kuriyama (*2); (*1 National Institute of Informatics, *2 School of Literature, Shirayuri College)

2 Research Papers

2.1 Cross-Lingual Information Retrieval Task

NTCIR-3 Chinese, Cross Language Retrieval Experiments Using PIRCS: K. L. Kwok; (Computer Science Department, Queens College, City University of New York)
Toshiba KIDS at NTCIR-3: Japanese and English-Japanese IR: Tetsuya Sakai, Makoto Koyama, Masaru Suzuki, and Toshihiko Manabe; (Knowledge Media Laboratory, Toshiba Corporate R&D Center)
Thomson Legal and Regulatory at NTCIR-3: Japanese, Chinese and English Retrieval Experiments: Isabelle Moulinier, Hugo Molina-Salgado, and Peter Jackson; (Thomson Legal and Regulatory, Research and Development Group)
Asian Language Parsing Evaluated by Hummingbird SearchServer^TMat NTCIR-3: Stephen Tomlinson; (Hummingbird)
KUNLP System for NTCIR-3 English-Korean Cross-Language Information Retrieval: Hee-Cheol Seo, Sang-Bum Kim, Baeg-Il Kim, Hae-Chang Rim, and Sang-Zoo Lee; (Department of Computer Science and Engineering, Korea University)
Deciding Indexing Strings with Statistical Analysis: Yoshiyuki TAKEDA, Kyoji UMEMURA and Eiko YAMAMOTO; (Toyohashi University of Technology)
Applying Multiple Characteristics and Techniques to Obtain High Levels of Performance in Information Retrieval: Masaki Murata, Qing Ma, and Hitoshi Isahara; (Communications Research Laboratory)
Different Retrieval Models and Hybrid Term Indexing: Robert W.P. LUK; (Department of Computing, Hong Kong Polytechnic University)
Description of NTU Approach to NTCIR3 Multilingual Information Retrieval: Wen-Cheng Lin, and Hsin-Hsi Chen; (Department of Computer Science and Information Engineering, National Taiwan University)
NTCIR-3 CLIR Experiments at MSRA: Hongzhao He (*1), and Jianfeng Gao (*2); (*1 Department of Computer Science and Engineering, Tianjin University, *2 Natural Language Computing Group, Microsoft Research)
CMU in Cross-Language Information Retrieval at NTCIR-3: Yiming Yang, and Nianli Ma; (Language Technologies Institute, Carnegie Mellon University)
ISCAS at NTCIR-3: Monolingual, Bilingual and MultiLingual IR Tasks: Junlin Zhang, Le Sun, Weimin Qu, Lin Du, Yufang Sun, Yangxing Fan, and Zhigen Lin; (Chinese Information Processing Center, Institute of Software, Chinese Academy of Sciences)
NTCIR-3 Cross-Language IR Experiments at ULIS: Atsushi Fujii (*1*2), and Tetsuya Ishikawa (*1); (*1 University of Library and Information Science, *2 CREST, Japan Science and Technology Corporation)
Chinese Language IR based on Term Extraction: Ji Donghong, Yang Lingpeng, and Nie Yu; (Laboratories for Information Technology)
Uniform Indexing and Retrieval Scheme for Chinese, Japanese, and Korean: Da-Wei Juang, and Yuen-Hsien Tseng; (Department of Library & Information Science, Fu Jen Catholic University)
Knowledge-light Asian Language Text Retrieval at the NTCIR-3 Workshop: Paul MCNAMEE; (Johns Hopkins University Applied Physics Laboratory)
NTCIR-3 CLIR Experiments at Osaka Kyoiku University -Comparison of Gram-based Indices-: Takashi SATO, and Koto HAN; (Osaka Kyoiku University)
OASIS at NTCIR-3: Monolingual IR Task: Vitaliy KLUEV; (The Core and Information Technology Center, The University of Aizu)
Waterloo at NTCIR-3: Using Self-supervised Word Segmentation: Xiangji Huang, Fuchun Peng, Dale Schuurmans, and Nick Cercone; (School of Computer Science, University of Waterloo)
POSNIR: Probabilistic Natural Language Information Retrieval System: Changki Lee, Seungwoo Lee, and Gary Geunbae Lee; (Department of Computer Science & Engineering, POSTECH)
Experiments on Cross-language and Patent Retrieval at NTCIR-3 Workshop: Aitao Chen (*1), and Fredric C. Gey (*2); (*1 School of Information Management and Systems, University of California at Berkeley, *2 UC Data Archive & Technical Assistance (UC DATA), University of California at Berkeley)
Simple Query Translation Methods for Korean-English and Korean-Chinese CLIR in NTCIR Experiments: Myung Gil Jang (*1*2), Pyung Kim (*1), Yun Jin*1, Suk-Hyun Cho (*1), and Sung Hyon Myaeng (*1); (*1 Department of Computer Science, Chungnam National University, *2 ETRI)

2.2 Patent Retrieval Task

Term Distillation for Cross-DB Retrieval: Hideo ITOH, Hiroko MANO, and Yasushi OGAWA; (Software R&D group, RICOH Co., Ltd.)
Experiment on Pseudo Relevance Feedback Method Using Taylor Formula at NTCIR-3 Patent Retrieval Task: Kazuaki KISHIDA; (Surugadai University / National Institute of Informatics)
NTCIR-3 PAT Experiments at Osaka Kyoiku University: Long Gram-based Index and Essential Words: Takashi SATO, Tomohiko SATOMOTO, and Koto HAN; (Osaka Kyoiku University)
NTT DTEC at Patent Retrieval Task: Yohichi Nakatani (*1), Koutarou Takada (*1), and Michihiro Isoda (*1), Manabu Okumura (*2), Makoto Iwayama (*2), Yuzo Marukawa (*2), and Akihiro Shinmori (*2); (*1 NTT DATA TECHNOLOGY CORPORATION, *2 Precision and Intelligence Laboratory, Tokyo Institute of Technology)
Patent Search: A Case Study of Cross-DB Associative Search: Yoshiki Niwa, Toru Hisamitsu, Shingo Nishioka, Osamu Imaichi, and Masakazu Fujio; (Central Research Laboratory, Hitachi, Ltd.)
NTCIR-3 Patent Retrieval Experiments at ULIS: Atsushi Fujii (*1*2), and Tetsuya Ishikawa (*1); (*1 University of Library and Information Science, *2 CREST, Japan Science and Technology Corporation)
Experiments on Cross-language and Patent Retrieval at NTCIR-3 Workshop: Aitao Chen (*1), and Fredric C. Gey (*2); (*1 School of Information Management and Systems, University of California at Berkeley, *2 UC Data Archive & Technical Assistance (UC DATA), University of California at Berkeley)
English-Japanese Cross-lingual Query Expansion Using Random Indexing of Aligned Bilingual Text Data: Magnus Sahlgren, Preben Hansen, and Jussi Karlgren; (Swedish Institute of Computer Science, SICS)
Using the Diff Command in Patent Documents: Masaki Murata, and Hitoshi Isahara; (Communications Research Laboratory)
Rhetorical Structure Analysis of Japanese Patent Claims using Cue Phrases: Akihiro SHINMORI (*1), Manabu OKUMURA (*2), Yuzo MARUKAWA (*2), Makoto IWAYAMA (*3),; (*1 Department of Computational Intelligence and Systems Sciences, Tokyo Institute of Technology / INTEC Web and Genome Informatics Corp., *2 Precision and Intelligence Laboratory, Tokyo Institute of Technology, *3 Precision and Intelligence Laboratory Tokyo Institute of Technology / Hitachi, Ltd.)

2.3 Question Answering Task

Towards Speech-Driven Question Answering: Experiments Using the NTCIR-3 Question Answering Collection: Tomoyosi Akiba (*1), Katunobu Itou (*1*3), Atsushi Fujii (*2*3), and Tetsuya Ishikawa (*2); (*1 National Institute of Advanced Industrial Science and Technology, *2 University of Library and Information Science, *3 CREST, Japan Science and Technology Corporation)
Oki QA System for QAC-1: Atsushi IKENO, and Hiroyuki OHNUMA; (Service Media Laboratory, Corporate Research & Development Center, Oki Electric Industry, Co., Ltd.)
Question and Answering System based on Predicate-Argument Matching: Daisuke Kawahara, Nobuhiro Kaji, and Sadao Kurohashi; (Graduate School of Information Science and Technology, University of Tokyo)
QUARK: A Question and Answering System using Newspaper Corpus as a Knowledge Source: Keizo KAWATA, Hiroyuki SAKAI, and Shigeru MASUYAMA; (Department of Knowledge-based Information Engineering, Toyohashi University of Technology)
SiteQ/J: A Question Answering System for Japanese: Seungwoo Lee, and Gary Geunbae Lee; (Department of Computer Science & Engineering, POSTECH)
A* Search Algorithm for Question Answering: Tatsunori MORI, Tomohiro OHTA, Katsuyuki FUJIHATA, and Ryutaro KUMON; (Graduate School of Environment and Information Sciences, Yokohama National University)
A Question-Answering System Using Unit Estimation and Probabilistic Near-Terms IR: Masaki Murata, Masao Utiyama, and Hitoshi Isahara; (Communications Research Laboratory)
NTCIR-3 QAC Experiments at Matsushita: Masako NOMOTO, Mitsuhiro SATO, and Hiroyuki SUZUKI; (Multimedia Systems Research Laboratory, Matsushita Electric Industrial Co., Ltd.)
NTT's QA Systems for NTCIR QAC-1: Yutaka Sasaki, Hideki Isozaki, Tsutomu Hirao, Koji Kokuryou, and Eisaku Maeda; (NTT Communication Science Laboratories, NTT Corp.)
Answer Extraction System by Question Type from Query-Biased Summary for Newspaper Articles: Yohei Seki; (Department of Informatics, The Graduate University for Advanced Studies, Sokendai / Department of Integrated Information Technology, Aoyama Gakuin University)
NYU/CRL QA System, QAC Question Analysis and CRL QA Data: Satoshi SEKINE (*1), Kiyoshi SUDO (*1), Yusuke SHINYAMA (*1), Chikashi NOBATA (*2), Kiyotaka UCHIMOTO (*2), and Hitoshi ISAHARA (*2); (*1 New York University, *2 Communications Research Laboratory)
Applying Structural Matching and Paraphrasing: Tetsuro TAKAHASHI (*1), Kozo NAWATA (*1), and Kentaro INUI (*1), and Shinya KOUDA (*2); (*1 Graduate School of Information Science, Nara Institute of Science and Technology, *2 Graduate School of Computer Science and System Engineering, Kyushu Institute of Technology)
NTT DATA Question-Answering Experiment at the NTCIR-3 QAC: Toru Takaki, and Yoshio Eriguchi; (Research and Development Headquarters, NTT DATA Corporation)
Exploitation of Newspaper-article Characteristics for Article Retrieval and Answer Extraction in QAC Task 2: Ruck THAWONMAS (*1), Takayuki TOMOIKE (*2), Tomohiko KAWACHI (*3), and Akio SAKAMOTO (*3); (*1 Department of Computer Science, Ritsumeikan University, *2 Course of Information Systems Engineering, Kochi University of Technology, *3 Department of Information Systems Engineering, Kochi University of Technology)
MAIMAI: A Question Answering System at NTCIR3 QAC-1: Fumito MASUI, and Masayuki MIYAGUCHI; (Department of Information Engineering, Faculty of Engineering, Mie University)
RitsQA: Ritsumeikan Question Answering System used for QAC-1: Jun'ichi FUKUMOTO, Tetsuya ENDO, and Tatsuhiro NIWA; (Ritsumeikan University)

2.4 Text Summarization Task

Text Summarization based on Itemized Sentences and Similar Parts Detection between Documents: Junichi FUKUMOTO; (Ritsumeikan University)
NTT/NAIST's Text Summarization Systems for TSC-2: Tsutomu Hirao (*1), Kazuhiro Takeuchi (*2), Hideki Isozaki (*1), Yutaka Sasaki (*1), and Eisaku Maeda (*1); (*1 NTT Communication Science Laboratories, NTT Corp., *2 Nara Institute of Science and Technology)
Trainable Automatic Text Summarization Using Segmentation of Sentence: Kai ISHIKAWA, Shin-ichi ANDO, Shin-ichi DOI, and Akitoshi OKUMURA; (Multimedia Research Laboratories, NEC Corporation)
Information Gain Ratio meets Maximal Marginal Relevance －A method of Summarization for Multiple Documents－: Tatsunori MORI, and Takuro SASAKI; (Graduate School of Environment and Information Sciences, Yokohama National University)
A Summarization System with Categorization of Document Sets: Chikashi NOBATA (*1), Satoshi SEKINE (*2), Kiyotaka UCHIMOTO (*1), and Hitoshi ISAHARA (*1); (*1 Computational Linguistics Group, Communications Research Laboratory, *2 Computer Science Department, New York University)
Two Different Summarization Methods at NTCIR3-TSC2: Coverage Oriented and Focus Oriented: Naoaki OKAZAKI (*1), Yutaka MATSUO (*2), Naohiro MATSUMURA (*1), Hironori TOMOBE (*1), and Mitsuru ISHIZUKA (*1); (*1 Graduate School of Information Science and Technology, The University of Tokyo, *2 Cyber Assist Research Center, AIST Tokyo Waterfront)
Unsupervised Acquisition of Knowledge about the Abbreviation Possibility of some of Multiple Phrases modifying the same Verb/Noun: Hiroyuki SAKAI, and Shigeru MASUYAMA; (Department of Knowledge-based Information Engineering, Toyohashi University of Technology)
Sentence Extraction by tf/idf and Position Weighting from Newspaper Articles: Yohei SEKI; (Department of Informatics, The Graduate University for Advanced Studies, Sokendai)

2.5 Web Retrieval Task

Evaluation of Web Retrieval Methods Using Anchor Text: Kenji TATEISHI, Hideki KAWAI, Susumu AKAMINE, Katsushi MATSUDA, and Toshikazu FUKUSHIMA; (Internet Systems Research Laboratories, NEC Corporation)
University of Tokyo/RICOH at NTCIR-3 Web Retrieval Task: Masashi TOYODA (*1), Masaru KITSUREGAWA (*1), Hiroko MANO (*2), Hideo ITOH (*2), and Yasushi OGAWA (*2); (*1 Institute of Industrial Science, University of Tokyo, *2 Software R&D Group, R ICOH Company, Ltd.)
Study on Merging Multiple Results from Information Retrieval System: Hiromi itoh OZAKU (*1*2), Masao UTIYAMA (*1), Hitoshi ISAHARA (*1), Yasuyuki KONO (*2), and Masatsugu KIDODE (*2); (*1 Communications Research Laboratory, *2 Nara Institute of Science and Technology)
Web Search Experiments Using OASIS: Vitaliy KLUEV; (The Core and Information Technology Center, The University of Aizu)
NTCIR-3 WEB Experiments at Osaka Kyoiku University －Towards Index Partitioning and Parallel Retrieval－: Takashi SATO, Yukikazu KYO, and Kihei KOBATA; (Osaka Kyoiku University)
Evaluating Speech-Driven IR in the NTCIR-3 Web Retrieval Task: Atsushi Fujii (*1*3) , and Katunobu Itou (*2*3); (*1 University of Library and Information Science, *2 National Institute of Advanced Industrial Science and Technology, *3 CREST, Japan Science and Technology Corporation)
A Experiment Report about a Web Information Retrieval System for 3rd NTCIR Web Task: Iwao NAGASHIRO (*1) , and Dafeng CAO (*2); (*1 Department of Information and Network, Tokai University, *2 Beijing Center for Japanese Studies)

3 Invited Speech

The Development and Evolution of TREC and DUC: Donna Harman; (Retrieval Group, Information Access Division, Information Technology Laboratory, National Institute of Standards and Technology)
Web IR Research: Can we do it without the data?: Amit Singhal; (Google, inc.)

Appendix

Topics / Questions

Cross-Lingual Information Retrieval Task :

Sample list
Full list
- 1994 Document set
  English : Chinese : Korean : Japanese
  1998-99 Document sets
  English : Chinese : Korean : Japanese

Patent Retrieval Task :

Full list (Formal Run Topics)

Japanese : Chinese (Traditional) : Chinese (Simplified) : Korean : English

Question Answering Task :

Sample list

Full list (Formal Run Questions)

Task 1 (Japanese : English)
Task 2 (Japanese : English )
Task 3 (Japanese : English )

Web Retrieval Task :

Sample list
Full list Japanese : English

Evaluation Results

NTCIR Workshop 3 Meeting CD-ROM
（Contents of CD-ROM delivered at the NTCIR-3 Workshop Meeting. For the topics / questions, use ones listed above.)

[Top of this page] [NTCIR Home] [NII Home]

Prepared by Keizo Oyama, NII

NTCIR Workshop3: Proceedings of the Third NTCIR Workshop on research in information Retrieval, Automatic Text Summarization and Question Answering (September 2001-October 2002)

TABLE OF CONTENTS