NTCIR-9 Workshop SpokenDoc Task Formal Run Evaluation Results

[NTCIR Home]

* Evaluation Results

• Evaluation results for the Spoken Term Detection (STD) subtask
  • For the CORE query set ( seven groups with 13 runs + one baseline run)
    • Overview results for the all submitted runs [SpokenDoc-STD-CORE-overview.pdf]
    • Evaluation results for each participating group and the baseline:
      • AKBL [SpokenDoc-STD-CORE-AKBL.xlsx]
      • ALPS [SpokenDoc-STD-CORE-ALPS.xlsx]
      • IWAPU [SpokenDoc-STD-CORE-IWAPU.xlsx]
      • NKGW [SpokenDoc-STD-CORE-NKGW.xlsx]
      • NKI11 [SpokenDoc-STD-CORE-NKI11.xlsx]
      • RYSDT [SpokenDoc-STD-CORE-RYSDT.xlsx]
      • YLAB [SpokenDoc-STD-CORE-YLAB.xlsx]
      • Baseline [SpokenDoc-STD-CORE-baseline.xlsx]
  • For the ALL query set ( two groups with five runs + one baseline run )
    • Overview results for the all submitted runs [SpokenDoc-STD-ALL-overview.pdf]
    • Evaluation results for each participating group and the baseline:
  • NKI11 [SpokenDoc-STD-ALL-NKI11.xlsx]
  • RYSDT [SpokenDoc-STD-ALL-RYSDT.xlsx]
  • Baseline [SpokenDoc-STD-ALL-baseline.xlsx]
    
  
  
• Evaluation results for the Spoken Document Retrieval (SDR) subtask
  • Lecture Retrieval Task ( four groups with 11 runs + baseline runs )
  • Evaluation results for each participating group and the baseline including result's overview [SpokenDoc-SDR-LEC.xlsx]
  • Passage Retrieval Task ( three groups with 10 runs + baseline runs )
  • Evaluation results for each participating group and the baseline including result's overview [SpokenDoc-SDR-PAS.xlsx]

 

These results were used in the overview paper:

T. Akiba, H. Nishizaki, K. Aikawa, T. Kawahara, and T. Matsui (2011) Overview of the IR for Spoken Documents Task in NTCIR-9 Workshop. In Proceedings of the 9 ^th NTCIR Workshop Meeting on Evaluation of Information Access Technologies, Tokyo, December 6-9, 2011.