Formal Run Results of Web Retrieval Task at the Third NTCIR Workshop (NTCIR-3)
Executives Committee of the NTCIR-3 Web Retrieval Task
ntcadm-web@nii.ac.jp
Last Modified: Sep 10, 2002
Introduction
(PDF format)
Overview of Systems Description
(PDF format)
Evaluation Result Sheets
(PDF format)
The Large-scale (100 Gigabytes)
Task I-A1: Survey Retrieval Tasks --Topic Retrieval Task--
Task I-A2: Survey Retrieval Tasks --Similarity Retrieval Task--
Task I-B: Target Retrieval Task
The Small-scale (10 Gigabytes)
Task II-A1: Survey Retrieval Tasks --Topic Retrieval Task--
Task II-A2: Survey Retrieval Tasks --Similarity Retrieval Task--
Task II-B: Target Retrieval Task
Summary of Evaluation Results --Graphs--
(PDF format)
The Large-scale (100 Gigabytes)
Task I-A1: Survey Retrieval Tasks --Topic Retrieval Task--
Task I-A2: Survey Retrieval Tasks --Similarity Retrieval Task--
Task I-B: Target Retrieval Task
The Small-scale (10 Gigabytes)
Task II-A1: Survey Retrieval Tasks --Topic Retrieval Task--
Task II-A2: Survey Retrieval Tasks --Similarity Retrieval Task--
Task II-B: Target Retrieval Task
Topics
(text format)
Lists of Queries and the Relevance Assessment
(text format accordings to 'trec_eval')
The Large-scale (100 Gigabytes)
The Small-scale (10 Gigabytes)
Introduction
(PDF format)
Overview of Systems Description
(PDF format)
Evaluation Result Sheets (PDF format)
The Large-scale (100 Gigabytes)
Task I-A1: Survey Retrieval Tasks --Topic Retrieval Task--
GRACE-LA1-1
GRACE-LA1-2
GRACE-LA1-3
GRACE-LA1-4
K3100-05
K3100-06
K3100-07
K3100-08
NAICR-I-A1-1
NAICR-I-A1-2
NAICR-I-A1-3
NAICR-I-A1-4
OKSAT-WEB-F-02
OKSAT-WEB-F-04
OKSAT-WEB-F-06
ORGREF-LA1-1
(organizers')
ORGREF-LA1-2
(organizers')
ORGREF-LA1-3
(organizers')
ORGREF-LA1-4
(organizers')
ORGREF-LA1-5
(organizers')
ORGREF-LA1-6
(organizers')
UAIFI1
UAIFI2
UAIFI3
UAIFI4
Task I-A2: Survey Retrieval Tasks --Similarity Retrieval Task--
GRACE-LA2-1
GRACE-LA2-2
GRACE-LA2-3
GRACE-LA2-4
NAICR-I-A2-1
NAICR-I-A2-2
NAICR-I-A2-3
NAICR-I-A2-4
Task I-B: Target Retrieval Task
GRACE-LB-1
GRACE-LB-2
GRACE-LB-3
GRACE-LB-4
K3100-13
K3100-14
K3100-15
K3100-16
NAICR-I-B-1
NAICR-I-B-2
NAICR-I-B-3
NAICR-I-B-4
ORGREF-LB-1
(organizers')
ORGREF-LB-2
(organizers')
ORGREF-LB-3
(organizers')
ORGREF-LB-4
(organizers')
ORGREF-LB-5
(organizers')
ORGREF-LB-6
(organizers')
UAIFI5
UAIFI6
UAIFI7
UAIFI8
The Small-scale (10 Gigabytes)
Task II-A1: Survey Retrieval Tasks --Topic Retrieval Task--
GRACE-SA1-1
GRACE-SA1-2
GRACE-SA1-3
GRACE-SA1-4
K3100-01
K3100-02
K3100-03
K3100-04
NAICR-II-A1-1
NAICR-II-A1-2
NAICR-II-A1-3
NAICR-II-A1-4
OASIS11
OASIS12
OKSAT-WEB-F-01
OKSAT-WEB-F-03
OKSAT-WEB-F-05
ORGREF-SA1-1
(organizers')
ORGREF-SA1-2
(organizers')
ORGREF-SA1-3
(organizers')
ORGREF-SA1-4
(organizers')
ORGREF-SA1-5
(organizers')
ORGREF-SA1-6
(organizers')
UAIFI10
UAIFI11
UAIFI12
UAIFI9
Task II-A2: Survey Retrieval Tasks --Similarity Retrieval Task--
GRACE-SA2-1
GRACE-SA2-2
GRACE-SA2-3
GRACE-SA2-4
NAICR-II-A2-1
NAICR-II-A2-2
NAICR-II-A2-3
NAICR-II-A2-4
Task II-B: Target Retrieval Task
GRACE-SB-1
GRACE-SB-2
GRACE-SB-3
GRACE-SB-4
K3100-09
K3100-10
K3100-11
K3100-12
NAICR-II-B-1
NAICR-II-B-2
NAICR-II-B-3
NAICR-II-B-4
ORGREF-SB-1
(organizers')
ORGREF-SB-2
(organizers')
ORGREF-SB-3
(organizers')
ORGREF-SB-4
(organizers')
ORGREF-SB-5
(organizers')
ORGREF-SB-6
(organizers')
UAIFI13
UAIFI14
UAIFI15
UAIFI16
Summary of Evaluation Results --Graphs-- (PDF format)
The Large-scale (100 Gigabytes)
Task I-A1: Survey Retrieval Tasks --Topic Retrieval Task--
`automatic' and `TITLE-only' runs
Recall-precision curves without considering links
Recall-precision curves with considering links
DCG curves without considering links
DCG curves with considering links
`automatic' and `DESC-only' runs
Recall-precision curves without considering links
Recall-precision curves with considering links
DCG curves without considering links
DCG curves with considering links
Task I-A2: Survey Retrieval Tasks --Similarity Retrieval Task--
`automatic' and `TITLE-and-RDOC[1]' runs
Recall-precision curves without considering links
Recall-precision curves with considering links
DCG curves without considering links
DCG curves with considering links
Task I-B: Target Retrieval Task
`automatic' and `TITLE-only' runs
DCG curves without considering links
DCG curves with considering links
`automatic' and `DESC-only' runs
DCG curves without considering links
DCG curves with considering links
The Small-scale (10 Gigabytes)
Task II-A1: Survey Retrieval Tasks --Topic Retrieval Task--
`automatic' and `TITLE-only' runs
Recall-precision curves without considering links
Recall-precision curves with considering links
DCG curves without considering links
DCG curves with considering links
`automatic' and `DESC-only' runs
Recall-precision curves without considering links
Recall-precision curves with considering links
DCG curves without considering links
DCG curves with considering links
Task II-A2: Survey Retrieval Tasks --Similarity Retrieval Task--
`automatic' and `TITLE-and-RDOC[1]' runs
Recall-precision curves without considering links
Recall-precision curves with considering links
DCG curves without considering links
DCG curves with considering links
Task II-B: Target Retrieval Task
`automatic' and `TITLE-only' runs
DCG curves without considering links
DCG curves with considering links
`automatic' and `DESC-only' runs
DCG curves without considering links
DCG curves with considering links
Topics (text format)
The Topics Delivered (EUC coding)
The Topics Delivered (SJIS coding)
The English-translated Topics
Lists of Queries and the Relevance Assessment (text format accordings to 'trec_eval')
The Large-scale (100 Gigabytes)
Without considering links
Relevant Level 0 (H)
Relevant Level 1 (H or A; Rigid)
Relevant Level 2 (H, A or B; Relaxed)
With considering links
Relevant Level 0 (H)
Relevant Level 1 (H or A; Rigid)
Relevant Level 2 (H, A or B; Relaxed)
The Small-scale (10 Gigabytes)
Without considering links
Relevant Level 0 (H)
Relevant Level 1 (H or A; Rigid)
Relevant Level 2 (H, A or B; Relaxed)
With considering links
Relevant Level 0 (H)
Relevant Level 1 (H or A; Rigid)
Relevant Level 2 (H, A or B; Relaxed)