NTCIR Project
Tools
NTCIREVAL

[JAPANESE] [NTCIR Home] [NTCIR Tools Home]


NTCIREVAL by Tetsuya Sakai

NTCIREVAL is a toolkit for computing various retrieval effectiveness metrics.
It can be used for NTCIR and TREC ad hoc retrieval evaluation, diversified search evaluation, NTCIR-8 Community QA Task evaluation and so on.

NTCIREVAL can compute metrics such as:
-Average Precision
-Q-measure
-nDCG
-Expected Reciprocal Rank (ERR)
-Graded Average Precision (GAP)
-Rank-Biased Precision (RBP)
-Normalised Cumulative Utility (NCU)
-Condensed-List versions of the above metrics
-Bpref
-D#-measures and DIN#-measures for diversity evaluation
-Intent-Aware (IA) metrics and P+Q# for diversity evaluation

For details, please refer to the README file included in the tar file.

The following NTCIR tasks used NTCIREVAL or its early version:
- NTCIR-7 and NTCIR-8 IR4QA
- NTCIR-8 and NTCIR-9 GeoTime
- NTCIR-8 Community QA
- NTCIR-9 and NTCIR-10 INTENT

Download

http://research.nii.ac.jp/ntcir/tools/NTCIREVAL.130507.tar.gz (makefile fixed on 2013-05-07)
http://research.nii.ac.jp/ntcir/tools/NTCIREVAL.120718.tar.gz (bug in the NEVIAPQ2sharpnev script fixed on 2012-07-18)
http://research.nii.ac.jp/ntcir/tools/NTCIREVAL.120528.tar.gz (bug for computing 1CLICK T-measure fixed on 2012-05-28)
http://research.nii.ac.jp/ntcir/tools/NTCIREVAL.120508.tar.gz (updated on 2012-05-10)
http://research.nii.ac.jp/ntcir/tools/NTCIREVAL.110426.tar.gz (updated on 2011-04-26)
http://research.nii.ac.jp/ntcir/tools/NTCIREVAL.100728.tar.gz (updated on 2010-07-28)


[JAPANESE] [NTCIR Home] [NTCIR Tools Home]

Updated on : 2013-07-03

ntc-admin