NTCIR-13 Conference


NTCIR-13 Conference Keynote

Date: December 6th (Wed), 2017 (Time: 10:00 a.m. - 11:00 a.m.)
--> Location: Hitotsubashi-hall, NII, Tokyo, Japan

Title: The practice of crowdsourcing: things to know about using humans and machines for labeling

Speaker: Omar Alonso (Microsoft)

Mr. Omar Alonso


Many data science applications that use machine learning techniques depend on humans providing the initial data set so algorithms can process the rest or to evaluate the performance of such algorithms. Not only can labeled data for training and evaluation be collected faster, cheaper, and easier than ever before, but we now see the emergence of novel infrastructure that combines computations performed by humans and machines. Building these labeling pipelines remain difficult and these difficulties need to be addressed by practitioners and researchers to advance the state of the art. In this talk, I’ll outline things that work in practice and describe a number of trade-offs when designing and implementing computation systems that use humans and machines.


Omar is a Principal Data Scientist Lead at Microsoft in Silicon Valley where he works on the intersection of social networks, temporal information, knowledge graphs, and human computation. He has shipped many features for Bing and other Microsoft properties. He is the co-chair for the new IR system-oriented conference, called DESIRES.


NTCIR 20th Anniversary Session

NTCIR 20th Anniversary Session: Invited Talk 1

Date: December 6th (Wed), 2016 (Time: 18:00 p.m. - 18:30 p.m.)
Location: Hitotsubashi-hall, NII, Tokyo, Japan

Title: NTCIR from the Beginning: A Personal Research Journey

Speaker: Fredric C Gey, University of California, Berkeley

Prof. Fredric C Gey


My participation in the first NTCIR workshop starting in 1997 led to a fifteen year research journey. It was a journey through Kanji, Katakana, Hiragana, segmentation, bigrams, phonetic recognition, decompounding and parallel corpora alignment for lexicon development. In languages the travels went through Japanese, Chinese, Korean, European languages, Russian, Arabic, and Hindi, as well as mathematics. It produced three SIGIR workshops (2002, 2006, 2009) on cross-language search and multilingual information access as well as a special issue of Information Processing and Management on cross language information retrieval. In later years the research direction changed toward geographic and geo-temporal information retrieval evaluation. This talk will cover highlights of my personal research journey and pay tribute to colleagues and students with whom I have been fortunate enough to collaborate.


After receiving a Master's degree in Mathematics from UC Berkeley n 1964, Fredric Gey worked for 3 1/2 years at Bell Laboratories (later ATT Labs Research). In 1967 he returned to Berkeley and worked for 21 years as a staff scientist at Lawrence Berkeley Laboratory in the Computer Science Research Department. In 1989 he returned to the Berkeley campus as Data Archivist and Librarian for Social Science and Health Statistics for UC Berkeley, while simultaneously pursuing a PhD in Information Science, which was conferred in 1993. His dissertation "Probabilistic Dependence and Logistic Inference in Information Retrieval" developed the first of several logistic regression ranking models which have stood the test of time. In 1996 he received a USA National Science Foundation Grant to develop logistic regression search models. In 1998 he turned the research direction of this grant toward cross-language information retrieval, leading to participation in the first NTCIR workshop in 1997-1999, and in 2000 to CLEF, the Cross Language Evaluation Forum for European languages. In addition to multilingual information access, he has also done research and development in nuclear forensics, geographic information retrieval, digital humanities and social science information systems. He was the General Chair of ACM-SIGIR 1999, the 22nd International Conference on Research and Development in Information Retrieval.

NTCIR 20th Anniversary Session: Invited Talk 2

Date: December 6th (Wed), 2017 (Time: 18:30 p.m. - 19:00 p.m.)
Location: Hitotsubashi-hall, NII, Tokyo, Japan

Title: NTCIR in the World: Two Decades of Impact

Speaker: Douglas W. Oard, University of Maryland (USA)

Prof. Douglas W. Oard


In the last part of the twentieth century, the idea of shared task evaluation emerged as a significant force shaping information retrieval research. In this talk, I will start by tracing the evolution of that idea at it moved around the world from its inception at Cambridge through its incubation at the Text Retrieval Conference, to its present incarnation at NTCIR. I’ll then look back over the history of NTCIR to highlight some of the impactful and innovative evaluation tasks that have been invented there, exploring the impact of each from national, regional, or global perspectives. I’ll wrap up with a few remarks on how the role of shared task evaluation in information retrieval research is evolving today, and what that might suggest for the future impact of NTCIR.


Douglas Oard is a Professor at the University of Maryland, College Park (USA), with joint appointments there in the College of Information Studies (Maryland’s iSchool) and the University of Maryland Institute for Advanced Computer Studies (UMIACS). He is also a visiting professor at the National Institute of Informatics (Japan) and a former general chair of NTCIR. Dr. Oard’s research interests center around the use of emerging technologies to support information seeking by end users. Additional information is available at http://terpconnect.umd.edu/~oard/.

Last modified: 2017-11-13