NTCIR

Those of us who work on Information Retrieval (IR) share several common perspectives that shape our work. First, we assume that the searcher knows what they are looking for. Well, we know that’s not always true, but it’s been a useful assumption. Second, we assume that what we’re searching for has some digital representation, whether it was born digital, digitized, or simply described using digital metadata. The NTCIR-18 SUSHI task this year makes it clear that’s not always true, but that’s not the first such case. Third, we assume that if an IR system can find things, that its job is to provide them. But of course there are cases in which some searchers should not see some things. Fourth, we assume that the searcher can recognize what they are looking for; that’s why we often cast our task as creating a ranked list. But we know that’s not always true, as in (for example) spoken interaction, where ranked lists aren’t too helpful. Fifth, we expect that the searcher will be satisfied once they have found what they are looking for. But we know from our work on Cross-Language IR, for example, that there are cases in which we can find things that the searcher simply can’t make sense of. In this talk, I will argue that there is much to be learned from these kinds of cases where our common perspectives break down, and I will suggest that in our new era of generative large language models it can be useful to think broadly about these kinds of questions.

Douglas W. Oard is a Professor and the incoming Interim Dean at the University of Maryland College of Information. Over his three decades of research in information retrieval he has worked on cross-language IR, speech retrieval, and search among sensitive content, among other “off the beaten path” topics; essentially this is a talk that has been 30 years in the making. And as a Visiting Professor at NII since 2014, he has been pleased to contribute to the development of NTCIR as one of the world’s premiere evaluation venues for information access technologies.