NTCIR-17 Conference

The 17th NTCIR Conference
Evaluation of Information Access Technologies

December 12(Tue) - 15(Fri), 2023
National Institute of Informatics (NII)*


The NTCIR-17 Conference ended successfully.
Thank you very much for your participation, cooperation, support, and interest.

Online Proceedings

Photo AlbumsNEW

DAY-0 (December 11): Pre-NTCIR Seminar
[ Pre-NTCIR Seminar ]
DAY-1 (December 12): Tutorial & EVIA 2023
[ Tutorial & EVIA 2023 ]
DAY-2 (December 13): NTCIR-17
[ Welcome & Overview, Keynote1, Task Overview, Panel Session ]
DAY-3 (December 14): NTCIR-17
[ Task Sessions, Breakout Session, Poster & Lunch, Banquet ]
DAY-4 (December 15): NTCIR-17
[ Task Session, Keynote 2, Poster & Lunch, Invited Talk, Closing, Awards & Wrapup ]

The Seventeenth NTCIR conference (NTCIR-17) would like to welcome all of you who are interested in and/or working on information access research. This sesquiannual event was first held in 1999, and has grown steadily ever since. About 303 people from 35 different countries attended NTCIR-16 and we are hoping that the event will be even more successful this time. NTCIR covers diverse information access tasks including, but not limited to, information retrieval, question answering, text summarisation and text mining, with an emphasis on East Asian languages and English.

At the NTCIR-17 conference, we will hear exciting reports from the following tasks:

- FairWeb-1 ("FairWeb-1")
- Fine-grained Argument Understanding in Financial Analysis ("FinArg-1")
- Medial Natural Language Processing for Social media and Clinical texts ("MedNLP-SC")
- Personal Lifelog Organisation & Retrieval ("Lifelog-5")
- QA Lab for Political Information-4 ("QA Lab-PoliInfo-4")
- Resource Transfer Based Dense Retrieval ("Transfer")
- Session Search ("SS-2")
- Unbiased Learning to Rank Evaluation Task 2 ("ULTRE-2")
- Understanding of non-Financial Objects in Financial Reports ("UFO")

While the NTCIR conference is a venue for task participants and organisers to discuss the official task results, open problems and future task designs, our door is open even to those who did not participate in any of the NTCIR-17 tasks. For the previous Conference, among them about half were researchers participated in the "task(s)" set by NTCIR, and others contributed to fruitful discussion.

NTCIR-17 will be co-located with EVIA 2023. EVIA is a one-day workshop for information access researchers to present their work on evaluation methods, measures, test collections, experimental designs and so on.

We look forward to seeing you all at NII in December!

Call For Conference Participation

Keynote & Invited Talk

Keynote 1

  • Date: DAY-2, Dec 13th (Wed), 2023 (Time: 10:45 - 11:45)
  • Speaker: Paul Thomas (Microsoft)
  • Title: Using language models for relevance labelling

  • Keynote 2
  • Date: DAY-4, Dec 15th (Dec), 2023 (Time: 11:00 - 12:00)
  • Speaker: Fernando Diaz (CMU, USA)
  • Title: The Principles of Depth, Breadth, and Precarity in Developing Metrics for Information Retrieval Evaluation

  • EVIA Keynote
  • Date: DAY-1, Dec 12th (Tue), 2023 (Time: 13:10 - 13:40)
  • Speaker: Ian Soboroff (NIST, USA)
  • Title: Evaluating Systems that Generate Content.

  • Tutorial
  • Date: DAY-1, Dec 12th (Tue), 2023 (Time: 10:45 - 11:30)
  • Speaker: Tetsuya Sakai (Waseda University)
  • Title: A Practical Guide to Computing Evaluation Measures and Comparing Systems: Twelve Small Tips

  • Panel
  • Date: DAY-2, Dec 13th (Wed), 2023 (Time: 16:00 - 16:45)
  • Panelists: Haruka Maeda (Kyoto University), Paul Thomas (Microsoft), and Mark Sanderson (RMIT University)
  • Moderator: Tetsuya Sakai (Waseda University)
  • Panel: Responsible Information Access: Fairness, Harmlessness, Sustainability, and More

  • EVIA Panel
  • Date: DAY-1, Dec 12th (Tue), 2023 (Time: 16:40 - 17:20)
  • Panelists: Akiko Aizawa (NII), Inho Kang (Naver), Yiqun Liu (Tsinghua University), Paul Thomas (Microsoft)
  • Moderator: Doug Oard
  • Panel: Evaluation of Large Language Models

  • Pre-NTCIR Seminar
  • Date: DAY-0, Dec 11th (Mon), 2023 (Time: 16:30 - 18:00)
  • Speaker: Mark Sanderson (RMIT University)
  • Title: Three decades of web search, how goes the retrieval revolution?
  • Program

    Sponsors for NTCIR-17 Conference

    Details will be announced at a later date.

    Institutional Supporters

    Thank you to our sponsors!

    Contact Us

    For further information, please contact NTCIR Project Office (Conference) at

    Thank you very much for your participation, cooperation, support, and interest.

    NTCIR-17 Program Co-Chairs:
    Takehiro Yamamoto (University of Hyogo, Japan)
    Zhicheng Dou (Renmin University of China)
    NTCIR-17 General Co-Chairs:
    Charles Clarke (University of Waterloo, Canada)
    Noriko Kando (National Institute of Informatics, Japan)
    Makoto P. Kato (University of Tsukuba, Japan)
    Yiqun Liu (Tsinghua University, China)

    Last Modified: 2024-01-17