NTCIR-13 QA Lab-3

Phase-1 Formal Run is ongoing

Registration deadline: Until Monday, May 1, 2017, SST

Task Important dates Register here

Contents

Important Dates

Time Zone:
SST (Samoa Standard Time, UTC-11)
JST (Japan Standard Time, UTC+9)

1st Competitive Runs (Japanese Subtask only)

WORKS START DATE   END DATE
Mock exam training data distribution Monday, Aug 22, 2016
at 0:00 a.m. JST
Todai mock exam Friday, Sep 30, 2016
at 0:00 a.m. JST		 Monday, Oct 10, 2016
at 11:59 p.m. JST

Phase-1 (English and Japanese Subtasks)

End-to-end subtask for Essay questions

WORKS START DATE   END DATE
End-to-end subtask formal run Thursday, Feb 9, 2017
at 0:00 a.m. SST		 Monday, Feb 13, 2017
at 11:59 p.m. SST
Evaluation for extraction Thursday, Mar 2, 2017
at 0:00 a.m. SST		 Wednesday, Mar 8, 2017
at 11:59 p.m. SST
Evaluation for end-to-end and summarization Thursday, Mar 9, 2017
at 0:00 a.m. SST		 Wednesday, Mar 22, 2017
at 11:59 p.m. SST

Extraction subtask for Essay questions

WORKS START DATE   END DATE
Extraction subtask formal run Thursday, Feb 9, 2017
at 0:00 a.m. SST		 Monday, Feb 13, 2017
at 11:59 p.m. SST
Evaluation for extraction Thursday, Mar 2, 2017
at 0:00 a.m. SST		 Wednesday, Mar 8, 2017
at 11:59 p.m. SST
Evaluation for end-to-end and summarization Thursday, Mar 9, 2017
at 0:00 a.m. SST		 Wednesday, Mar 22, 2017
at 11:59 p.m. SST

Summarization subtask for Essay question

WORKS START DATE   END DATE
Summarization subtask formal run Thursday, Feb 16, 2017
at 0:00 a.m. SST		 Monday, Feb 20, 2017
at 11:59 p.m. SST
Evaluation for extraction Thursday, Mar 2, 2017
at 0:00 a.m. SST		 Wednesday, Mar 8, 2017
at 11:59 p.m. SST
Evaluation for end-to-end and summarization Thursday, Mar 9, 2017
at 0:00 a.m. SST		 Wednesday, Mar 22, 2017
at 11:59 p.m. SST

Evaluation method subtask for Essay questions

WORKS START DATE   END DATE
Evaluation method subtask formal run Thursday, Feb 23, 2017
at 0:00 a.m. SST		 Wednesday, Mar 1, 2017
at 11:59 p.m. SST
Evaluation for extraction Thursday, Mar 2, 2017
at 0:00 a.m. SST		 Wednesday, Mar 8, 2017
at 11:59 p.m. SST
Evaluation for end-to-end and summarization Thursday, Mar 9, 2017
at 0:00 a.m. SST		 Wednesday, Mar 22, 2017
at 11:59 p.m. SST

End-to-end subtask for Term questions

WORKS START DATE   END DATE
End-to-end subtask formal run Thursday, Feb 2, 2017
at 0:00 a.m. SST		 Monday, Feb 6, 2017
at 11:59 p.m. SST

End-to-end subtask for Multiple-choice questions

WORKS START DATE   END DATE
End-to-end subtask formal run Thursday, Feb 2, 2017
at 0:00 a.m. SST		 Monday, Feb 6, 2017
at 11:59 p.m. SST

Phase-2 (English and Japanese Subtasks)

End-to-end subtask for Essay questions

WORKS START DATE   END DATE
End-to-end subtask formal run Thursday, May 18, 2017
at 0:00 a.m. SST		 Monday, May 22, 2017
at 11:59 p.m. SST
Evaluation for extraction Thursday, June 8, 2017
at 0:00 a.m. SST		 Wednesday, June 14, 2017
at 11:59 p.m. SST
Evaluation for end-to-end and summarization Thursday, June 15, 2017
at 0:00 a.m. SST		 Wednesday, June 21, 2017
at 11:59 p.m. SST

Extraction subtask for Essay questions

WORKS START DATE   END DATE
Extraction subtask formal run Thursday, May 18, 2017
at 0:00 a.m. SST		 Monday, May 22, 2017
at 11:59 p.m. SST
Evaluation for extraction Thursday, June 8, 2017
at 0:00 a.m. SST		 Wednesday, June 14, 2017
at 11:59 p.m. SST
Evaluation for end-to-end and summarization Thursday, June 15, 2017
at 0:00 a.m. SST		 Wednesday, June 21, 2017
at 11:59 p.m. SST

Summarization subtask for Essay questions

WORKS START DATE   END DATE
Summarization subtask formal run Thursday, May 25, 2017
at 0:00 a.m. SST		 Monday, May 29, 2017
at 11:59 p.m. SST
Evaluation for extraction Thursday, June 8, 2017
at 0:00 a.m. SST		 Wednesday, June 14, 2017
at 11:59 p.m. SST
Evaluation for end-to-end and summarization Thursday, June 15, 2017
at 0:00 a.m. SST		 Wednesday, June 21, 2017
at 11:59 p.m. SST

Evaluation method subtask for Essay questions

WORKS START DATE   END DATE
Evaluation method subtask formal run Thursday, June 1, 2017
at 0:00 a.m. SST		 Monday, June 5, 2017
at 11:59 p.m. SST
Evaluation for extraction Thursday, June 8, 2017
at 0:00 a.m. SST		 Wednesday, June 14, 2017
at 11:59 p.m. SST
Evaluation for end-to-end and summarization Thursday, June 15, 2017
at 0:00 a.m. SST		 Wednesday, June 21, 2017
at 11:59 p.m. SST

End-to-end subtask for Term questions

WORKS START DATE   END DATE
End-to-end subtask formal run Thursday, May 11, 2017
at 0:00 a.m. SST		 Monday, May 15, 2017
at 11:59 p.m. SST

End-to-end subtask for Multiple-choice questions

WORKS START DATE   END DATE
End-to-end subtask formal run Thursday, May 11, 2017
at 0:00 a.m. SST		 Monday, May 15, 2017
at 11:59 p.m. SST

2nd Competitive Runs (Japanese Subtask only)

TBA

Research run (English and Japanese Subtasks)

End-to-end subtask for Multiple-choice questions

WORKS START DATE   END DATE
End-to-end subtask formal run Thursday, July 6, 2017
at 0:00 a.m. SST		 Monday, July 10, 2017
at 11:59 p.m. SST

NTCIR-13 Events (details)

Friday, September 1, 2017 Evaluation results return
Friday, September 1, 2017 Task overview release (draft)
Sunday, October 1, 2017 Submission due of participant papers (draft)
Wednesday, November 1, 2017 Final submission due of all camera-ready papers (task overview papers / participant papers)
From Tuesday, December 5, 2017
Until Friday, December 8, 2017		 NTCIR-13 Conference & EVIA 2017 in NII, Tokyo, Japan