NTCIR-19
NTCIR-19 Aims
Since 1997, the NTCIR project has promoted research efforts for enhancing Information Access (IA) technologies such as Information Retrieval (IR), Text Summarization, Information Extraction (IE), and Question Answering (QA) techniques. Its general purposes are to:
- Offer research infrastructure that allows researchers to conduct a large-scale evaluation of IA technologies
- Form a research community in which findings based on comparable experimental results are shared and exchanged, and
- Develop evaluation methodologies and performance measures of IA technologies.
Collaborative works in the NTCIR allow us to create large-scale test collections that are indispensable for checking the effectiveness of novel IA techniques. In addition, in the process of the collaboration, it is expected that deep insight into research problems is successfully shared among researchers.
In particular, the NTCIR-19 focuses on three topics on IA technology mainly;
- Modern IR tasks such as instruction design for agentic search (AgenticInstruction), lifelog content retrieval (Lifelog-7), pre-trained model retrieval (ModelRetrieval), and known-item retrieval under uncertainty (Tip-of-the-Tongue).
- Response evaluation tasks such as automatic evaluation of LLMs' responses (AEOLLM-2), confidence-aware RAG (R2C2), and claim verification in scientific literature (SciClaimEval).
- Deep NLP tasks in specialized domains such as e-commerce, agricultural information, human values, finance, medicine, regulatory compliance, etc. (CAMEO, DAGRI, FEHU, FinArg-3, HIDDEN-RAD2, MedNLP-CALL, and RegCom)
In addition to continuing the established directions, NTCIR-19 introduces more tasks related to agentic AI, exploring how large language models can be guided, aligned, and evaluated as autonomous information access agents. We sincerely hope that NTCIR-19 will be beneficial to all research groups who consider their participation to advance their research and enjoy their research.