Submission Archives - DATA/User Agreement
(For Research Purposes)

[NTCIR home] [Test Collection]

---

Submission Archives consist of the results which were submitted by task participants of the NTCIR Workshop that NII organized,Evaluation Data and Evaluation Tools.

Submission Archives are the datasets to be used for research related to evaluation methods and metrics of information access systems. With regard to the purpose of use, please make contact with Noriko Kando, the NTCIR Project leader at ntc-admin@.

|ACLIA|CLIR on News|CLQA|CQA|INTENT|1CLICK|Opinion|Patent|Patent Mining|
|Patent Translation|RITE|SpokenDoc|WEB|

C:Chinese　(Ct:Traditional Chinese　 Cs: Simplified Chinese） E:English　J:Japanese　K:Korean

*1: English translation is available
*2: Removed a few topics from the formal run such that a very small number of relevant document has been returned.
*3: 300+200 questions for C documents, and 300+200 questions for JE documents
*4: Right, Unsupported, Wrong
*5: system response conceptually containing the nugget or not
*6: submissions for NTCIR-3Test Collection:162, for NTCIR-4:166, for NTCIR-5:166
*7: official runs (unofficial runs)
*8: obligatory runs (optional runs)
*9: One team submitted seven EN-CS runs but the sixth and the seventh runs were not used for pooling and are excluded from the analyses.
*10: gakkai subfiles: 1997-1999, kaken subfiles: 1986-1997
*11: Reference translation (J): 1381 sentences, Human Judgement: 100 sentences * 5 runs * 3 humans
*12: Reference translation (E): 1381 sentences + 300 sentences * 2 humans, Human Judgement: 100 sentences * 15 runs * 3 humans
*13: Reference translation (J): 1119 sentences
*14: Reference translation (E):1251 sentences + 300 sentences * 3 humans
*15: Some Japanese runs were not included in the pools for the official evaluations. For more details, please see this page.
*16: Reference translation(E): 2000 sentences, Human Judgement: 100 sentences ×(adequacy: 23 runs, acceptabiity: 13runs) ×3 humans, Development Data(C-E): 2000 sentence pairs
*17: Reference translation(J): 2000 sentences, Human Judgement: 100 sentences ×(adequacy:17 runs, acceptabiity: 11runs) ×3 humans, Development Data(E-J): 2000 sentence pairs
*18: Reference translation(E): 2000 sentences, Human Judgement: 100 sentences ×(adequacy: 19 runs, acceptabiity: 14runs) ×3 humans, Development Data(J-E): 2000 sentence pairs
*19: Cs: Development Data　407 sentence pairs, Test Data　407 sentence pairs, Ct: Development Data　421sentence pairs, Test Data　900 sentence pairs, J: Development Data　500 sentence pairs, Test Data　500 sentence pairs
*20: Cs: Development Data　407 sentence pairs, Test Data　407 sentence pairs, Ct: Development Data　421 sentence pairs, Test Data　900 sentence pairs, J: Development Data　440 sentence pairs, Test Data　440 sentence pairs
*21: Development Data　449 sentence pairs, Test Data　442 sentence pairs


Research Purpose Use of Submission Archives-In order to use the Submission Archives, please fill out the Application Form and send by E-mail to ntc-secretariat. Especially with regard to 'Purpose of Use' in the Application Form, please refer to the memorandum(Example) and inform us in detail and definitely. If you have any question or special request, please contact Noriko Kando, the NTCIR Project leader at ntc-admin@.


How to obtain Submission Archives --- The followings are the procedures to obtain the Submission Archives. The Submission Archives and data available from NII are free of charge.

• The application form of the Submission Archives must be filled out and sent by E-mail to ntc-secretariat. -->instructions
• After ｒeview in the NII, upon approval of your application, you will be contacted and asked to sign User Agreement (memorandum on Permission to Use) and send the form by postal mail or courier to the address below. We will inform you of more details upon approval.

• User Agreement (memorandum on Permission to Use Test Collection):Example

  
  Address
  　　　　NTCIR project (Rm.1309)
  　　　　National Institute of Informatics
  　　　　2-1-2 Hitotsubashi Chiyoda-ku, Tokyo
  　　　　102-8430, JAPAN
  
  PHONE: +81-3-4212-2750
  FAX: +81-3-4212-2751
  Email: ntc-secretariat
  

Research Activities Report/Publication Report

"Research Activities Report"" and "Publication Report" should be submitted by the users of NTCIR Submission Archive.

"Research Activities Report"
The form of Research Activities Report must be filled out and sent by E-mail to ntc-report more than one month before the expiration of the Memorandum.

"Publication Report related to NTCIR"--> please refer to the page "To Publication Report related to NTCIR"

Mailing List
The release of the new test collections and correction information shall be announced through the NTCIR Mailing list: ntcir
To subscribe the Mailing list, please refer this page.

NTCIR
Please refer this page about the ongoing NTCIR.

---

[NTCIR home] [Test Collection]