1. Collections

Participants are free to use any resources available with the exception of the answer sets (readily available online in Japanese). In addition, the following resources are provided, but are not required to be used. i) three sets of National Center Tests, ii) two sets of Second-stage Examinations, iii) Knowledge Sources (a snapshot of Wikipedia subset related to world history), and vi) Right Answers

Figure 1: Exam datasets of the university of Tokyo

Figure 2: Exam datasets of the National Center Test

1.1 Sets of National Center Tests

• Sets of National Center Tests, available in Japanese and English.

1.2 Sets of Second-stage Examinations

• Sets of Second-stage Examinations, available in Japanese and English.

1.3 Knowledge Sources

• Japanese high school textbooks on world history, available in Japanese.
• A snapshot of Wikipedia, available in Japanese and in English. (Participants can also use the current up-to-date version).
  • Solr Instance with Indexed Wikipedia Subset (available in English)
    https://github.com/oaqa/ntcir-qalab-cmu-baseline/wiki/Solr-Instance-with-Indexed-Wikipedia-Subset
  • NTCIR-11 QA Lab Japanese subtask: Wikipedia Data Set
    http://warehouse.ntcir.nii.ac.jp/openaccess/qalab/11QALab-ja-wikipediadata.html
• World history ontology, available in Japanese.
  イベントオントロジーEVT

1.4 Right Answers

• Right answers for National Center Tests, available in English and Japanese.
• Right answers for Second-stage Examinations, available in English and Japanese.
• Reference essays and nuggets for Essays, available in Japanese.

2. Tools

• 1 baseline QA system for English, based on UIMA (CMU)
  https://github.com/oaqa/ntcir-qalab-cmu-baseline
• 1 baseline QA system for Japanese, based on YNU's MinerVA, CMU's Javelin and a question analysis module by Madoka Ishioroshi, re-constructed and implemented as UIMA components by Yoshinobu Kano
  https://bitbucket.org/ntcirqalab/factoidqa-centerexam/
• Scorer and Format Checker for National Center Test
  https://bitbucket.org/ntcirqalab/qalabsimplescorer
• Passage Retrieval Engine passache
  https://code.google.com/p/passache/

Back to top