Submission raw data archive for research purpose
- To: email@example.com
Subject: [ntc5-workshop:2] Submission raw data archive for research purpose
From: Noriko Kando <kando>
Content-Type: text/plain; charset=ISO-2022-JP
Date: Mon, 22 Aug 2005 19:00:08 +0900
Dear NTCIR-5 participants,
Thank you again for your participating the NTCIR-5.
I'm writing this email to ask your approval for research purpose
use of the NTCIR-5 submission raw data. The "submission
raw data" means any system results that you submitted to the
NTCIR's task organizers or ntcir organizers. It includes for example
the retrieval results file or the list of answers that your system
A set of submission raw data is a very valuable resource for
the research on the evaluation metrics or evaluation
methodologies for information access technologies such as
IR or question answering, To investigate the new evaluation
metrics, we always have to show the robustness and
stability of the proposed metrics as well as how the proposed
metrics represent the effectiveness of each of the different
retrieval mechanisms based on the retrieval results produced
by various IR systems or QA systems on the same set of topics.
NTCIR submission raw data is one of such very valuable data and
it would be great contribution for research in information access
technologies if such valuable data will be available for research
purpose use for the researchers who seriously investigate
the evaluation of information access technologies such as IR, QA,
summarization, or so.
For the appropriate research purpose use of those submission raw
data, NII and the researchers who uses the data will exchange the
memorandum on the research purpose use of the data.
* NTCIR-5 SUBMISSION RAW DATA:
will be available for the active participants of the NTCIR-5 of the
task before the NTCIR-5 Meeting in December. Those who wish
to access to the Submission Raw Data of the task that you participated
at NTCIR-5, must submit the Memorandum (user agreement form).
After the NTCIR-5 Meeting, the NTCIR-5 submission raw data
archive will be available for research purpose for the non-participating
researchers. Those data will be only available after application
will be reviewed and approved, and having exchanged the memorandum.
* PLEASE APPROVE:
We shall greatly appreciate it if you will kindly understand the importance
of the research purpose using such submission data and approve that
NII will grant the research purpose use for those who will apply to NII
the research purpose use of the submission raw data.
We are grateful if you will let us know whether you will agree for such
research purpose use or not, by sending the form below to me, Noriko
Kando at kando, and the NTCIR Project Office at
If you will not hear from you by September 20, 2005, We will assume that
you agree for research purpose use by the above mentioned conditions.
Also if you participated past NTCIR workshops, please include the
answers for your submitted system results at the past NTCIR workshops.
The same due process (not responded by September 20, 2005, automatically
treated as "yes" will be applied to the past NTCIR submitted system results.
WHAT WE EXPECT:
Using the submission raw data of the task you participated
in NTCIR-5, we do expect that each of task participants
will join the task organizers to analyse the evaluation
results, or propose a new metrics or examine how the metrics
worked. NTCIR is not a competition but it is an evaluation.
We, the task participants and the organizers would like to
work together to see if our evaluation really can differentiate
the better systems from not-so-good systems and what kind of
lessons we can learned from the results. Please join us to
make the NTCIR more vital forum of the domain.
* COMMENTS and OPINIONS are WELCOME:
We will circulate the same message to the past ntcir
participants through ntcirjp mailing list later of
this week. Meanwhile, any opinion, support or advices are
more than welcome as well as the responce.
We asked the permission to use the submission raw data
for research purpose for the NTCIR-3 CLIR participants
on April 2004 and the data archive is alreadly avaialble
for research purpose.
Thank you for your consideration in advance,
Your participated NTCIR and task(s):
ex. NTCIR-1 adhoc, NTCIR-4 CLIR, etc.
Your group name and groups IDs.:
(1) YES, I AGREE to the research purpose use of the above mentioned
submission raw data that our group produced.
(2) NO, I DON'T AGREE to the research purpose use of
the submission raw data that our group submitted.