|
|||||
|
NTCIR-8 Workshop Meeting
http://www.ibm.com/deepqa Extended Abstract: On April 27, 2009, IBM unveiled the details of a project for building an
advanced computing system that will be able to compete with humans at the
game of Jeopardy! Computer systems that can directly and accurately answer
peoples' questions over a broad domain of human knowledge have been envisioned
by scientists and writers since the advent of computers themselves. Consider,
for example, the computer on Star Trek ? how it understands questions and
quickly provides accurate, customized answers and can engage in a fluent
information seeking dialog with the user. We call this technology open
domain question answering and it has tremendous promise for impacting society
and business. Applications in business intelligence, health care, customer
support, enterprise knowledge management, social computing, science and
government would all benefit from such technology. The Project Watson is
addressing a grand challenge in Computer Science aimed at illustrating
how the integration and advancement of Natural Language Processing (NLP),
Information Retrieval (IR), Machine Learning (ML), massively parallel computation
and Knowledge Representation and Reasoning (KR&R) can advance open-domain
automatic Question Answering to a point where it clearly and consistently
rivals the best human performance. An exciting proof-point in this challenge
is to develop a computer system that can successfully compete against top
human players at the well-known Jeopardy! quiz show. Attaining champion-level
performance at the game of Jeopardy! requires a computer system to rapidly
and accurately answer challenging open-domain questions, and to predict
its own performance on any given category/question. The system must deliver
high degrees of precision and confidence over a very broad domain with
a 3 second response time. It is highly unlikely that any system will be
able to clearly justify all the answer with perfect certainty over such
a broad range of natural language questions and content. Computing accurate
confidences is an important requirement for determining when to gbuzz inh
against your competitors and how much to bet. While critical for winning
the game, high precision and accurate confidence computations are just
as critical for a QA system to provide real value in business settings.
The need for speed and for very high precision demands a massively parallel
compute platform capable of generating and evaluating 1000fs of hypotheses
and their associated evidence. In this talk we will introduce the audience
to the Jeopardy! Challenge and describe our technical approach and progress
on this grand-challenge problem. EVIA 2010 Biography: John Nave is the Principal development Manager in Microsoft Search Technology Center Japan, leading Bing development for Japanese market. He has been developing and shipping many products that apply technology to bring benefit to Microsoftfs customers for the last 14 years. The very first of these products was a Japanese tokenizer, followed by natural language products for several languages. With extensive experience in Japan language processing and Japanese market, John is now focused on understanding the needs of customers in Japan and bringing new and more useful solutions to this market. Prior to Microsoft, John worked in finance and software start-up businesses in Japan and the US. He is a graduate of the University of Washington, and attended Keio University as a Monbusho invited scholar. (Many years ago!) NTCIR-8 Workshop Meeting
This talk will provide a brief summary of the most significant results achieved by CLEF in the past ten years, it will describe the new format and organization for CLEF which is being experimented for the first time in CLEF 2010, and it will discuss why scientific data should play a central role in the design and planning of an evaluation campaign and how large-scale evaluation campaigns could adopt interoperable infrastructures to foster the sharing and re-use of such data. Biography: Nicola Ferro is assistant professor in Computer Science at the Department of Information Engineering and at the Faculty of Statistical Sciences of the University of Padua, Italy. He teaches the courses on Digital Libraries, Information Retrieval, and Databases. He received a Ph.D. degree in Computer Science from University of Padua in 2005. He holds a Laurea degree from University of Padua in Telecommunications Engineering. His main research interests are digital libraries and archives, their architectures, interoperability, and evaluation, as well as multilingual information access and its evaluation. He is and has been involved in the overall coordination of the CLEF (Cross Language Evaluation Forum) evaluation campaigns since 2005. He is scientific leader of the DL.org working group on quality in digital libraries. He is programme co-chair of the CLEF 2010 Conference on Multilingual and Multimedia Information Access Evaluation. He has participated and participates in several national and international projects among which Europeana Connect (multilingual information access services for Europeana and their evaluation), Europeana v 1.0 (multilinguality and annotations in the Europeana Data Model), TrebleCLEF (best practices, collaboration, and evaluation for multilingual information access systems), TELplus (enhanchement of The European Library portal towards Europeana), SAPIR (search in audio visual content using peer-to-peer information retrieval), and DELOS (the European network of excellence on Digital Libraries). He has published more than 60 papers on digital library architectures, interoperability, and services; multilingual information access and its experimental evaluation; the management of the scientific data produced during evaluation campaigns. He is member of ACM and IEEE. Title: "ClueWeb09 and TREC Diversity"
For TREC 2009, topics for the track were created from the logs of a commercial search engine, with the aid of tools developed at Microsoft Research. Given a target query, these tools extracted and analyzed groups of related queries, using co-clicks and other information, to identify clusters of queries that highlight different aspects and interpretations of the target query. These clusters were employed by NIST for topic development. For use by the diversity task, each resulting topic is structured as a representative set of subtopics, each related to a different user need. Documents were judged with respect to the subtopics, as well as with respect to the topic as a whole. In 2009, a total of 18 groups submitted runs to the diversity task. To evaluate these runs, the task used two primary effectiveness measures: ?-nDCG as defined by Clarke et al. (SIGIR 2008) and an gintent awareh version of precision, based on the work of Agrawal et al. (WSDM 2009). Developing and validating metrics for diversity tasks continues to be a goal of the track. For TREC 2010, we will report a number of additional evaluation measures that have been proposed over the past year, including an intent aware version of the ERR measure described by Chapelle et al. (CIKM 2009). Nick Craswell from Microsoft serves as the track co-coordinator. Ian Soboroff is the NIST contact. The ClueWeb09 collection was created through the efforts of Jamie Callan and Mark Hoy at the Language Technologies Institute, Carnegie Mellon University. More information may be found on the track Web page: http://plg.uwaterloo.ca/~trecweb/2010.html. Biography: Charles Clarke is a professor in the David R. Cheriton School of Computer Science at the University of Waterloo, Canada. He has published on a wide range of topics within the area of information retrieval, including papers related to evaluation, efficiency, ranking, parallel systems, security, question answering, document structure, and XML. He was a Program Co-Chair of SIGIR 2007 and General Co-Chair of SIGIR 2003. From 2004 to 2006 he was the coordinator of the TREC Terabyte Retrieval track. Since 2009 he has been a co-coordinator of the TREC Web Track. He is a co-author of the book Information Retrieval: Implementing and Evaluating Search Engines (MIT Press, 2010). He has previously held software development positions at a number of computer consulting and engineering firms. In 2006 he spent a sabbatical at Microsoft, where he was involved in their search engine development effort. Title:"E-Commerce Data through Rakuten Data Challenge"
academia for research purpose. The data includes the followings: 1) market item data and item homepage data 2) Hotel data and its review data 3) Golf course data and its review data We are planning to hold Rekuten R&D Symposium in January, 2011, where one of the sessions will be dedicated to the R&D activities using the data. The data is planned to be distributed through ALAGIN and NII-IDR in July, 2010. Biography: Masahiro Sanjo graduated Tokyo univ. in 2003. Currently he is the senior-technologist of Rakuten Institute of Technology.@He is interested in the technology of image analysis as the internet services. Satoshi Sekine received Ph.D. at New York University in 1994. Currently he is the director of Rakuten Institute of Technology, New York, as well as Associate Research Professor at New York University. He is interested various fields in NLP, including Information Extraction, Linguistic Knowledge Acquisition, Language Analysis and so on. We are looking forward to seeing you soon! Last updated: June 07, 2010 |
||||
