Dear Community QA Pilot Task participants, Based on your feedbacks for our previous announce both to this mailing list and in individual emails, we decided to choose "Plan B", i.e. using the 1st (smaller) version of "Yahoo! Chiebukuro Corpus" with the following schedule. Please submit the necessary forms AS SOON AS POSSIBLE to obtain the necessary documents and resources. The details of the task description shall be discussed through this mailing list. 1. SCHEDULE: Plan B: Use the 1st (smaller) version regardless of the availability of 2nd version. Document release:    16 MAR 2010 Topic release:    30 MAR 2010 Results Submission:    6 APR 2010 Evaluation Result    Return: 20 APR 2010 Participants' draft papers            Due: 2 MAY 2010 Organizers' feedback on draft         papers: by 6 MAY 2010 Final Papers Due:    15 MAY 2010 NTCIR-8 Meeting 15-18 JUNE 2010 --------------------------------------- 2. HOW TO OBTAIN THE CORPUS AND TASK DATA: 2.1. To obtain the TASK DATA, please submit the user agreement forms to NTCIR Office; http://research.nii.ac.jp/ntcir/ntcir-ws8/permission/perm-en.html please download the form, make 2 copies of both-side print, fill names and other information, put signature, and send the both completed forms to us. NII will put counter-sign, keep one copy and return one copy for your retention. 2.2. To obtain the "Yahoo! Chiebukuro Corpus", Ver.1, please read the instruction available at the following URL carefully and submit the necessary documents AS SOON AS POSSIBLE, so that you will be able to obtain the dataset in time ! http://research.nii.ac.jp/ntcir/ntcir-ws8/permission/ntcir8Chiebukuro-yahoo-en.html Please note that the corpus will be delivered from NII's Informatics Research Data Repository Group (IDR), which is a section responsible to manage and distribute the corpus and other resources for research purpose. It will take longer than the procedure in NTCIR. So please submit the form very soon ! 2.3. Sample Question: http://research.nii.ac.jp/ntcir/ntcir-ws8/yahoo/sample-question 3. TASK According to the call for participation, we are planning the following tasks. (1) Main Task: Best Answer Estimation (2) Optional Task: Question Type Classification We welcome any other suggestions. For (1), the test set consists of randomly selected 1500 questions and their associated answers. These were selected from 15 top categories according to the number of questions in the whole corpus. In addition to the original "Best Answers" selected by the Askers who raised the Questions, 4 individual assessors selected "Correct Answers" for each of these 1500 questions individually. We will use these different annotations to evaluate the system estimation. The details of the evaluation method shall be discussed here in this mailing list. For (2), we use a classification scheme proposed by Kuriyama 2009. The test set is the same as (1). A sample corpus will be delivered. The sample corpus consists of 500 questions and their associate answers selected from 5 categories, and which were annotated by 3 assessors individually. Hope you will enjoy CQA :-) Daisuke, Tetsuya and Noriko ntcadm-yahoo (at) nii ac jp