NTCIR Project
Research Purpose Use of Test Collection

[NTCIR Home] [NTCIR Data Home]

NTCIR-14 STC-3 (Short Text Conversation Task)

Test Collection

Dialogue Quality and Nugget Detection (DQND) Subtasks

Recently, many reserachers are trying to build automatic helpdesk systems. However, there are very few methods to evaluate such systems.
In STC-3 NDDQ subtasks, we aim to explore methods to evaluate customer-helpdesk dialogues automatically. This dataset have the following features:

In NTCIR-14 STC3-NDDQ, we consider annotations ground truth, and participants are required to predict nugget type for each turn (Nugget Detection, or ND) and dialogue quality for each dialogue (Dialogue Quality, or DQ).

Task data

The data collection of STC3-DQND consists of the following files:

For more details, please refer to the readme.md file and Overview of the NTCIR-14 Short Text Conversation Task:Dialogue Quality and Nugget Detection Subtasks in NTCIR-14 online proceedings.

To obtain the test collection

The test collection of STC3-DQND is available for research purpose only:

  • Download the DQND dataset (Training + Test, 2.7 Mb)
  • Reference

    Contact us: ntc-secretariat

    [NTCIR Home] [Top of this page] [NTCIR Data Home]