ADDENDUM TO:
	The Effect of Topic Sampling on Sensitivity Comparisons of Information Retrieval Metrics
by Tetsuya Sakai

This paper examined three methods for creating two sets of topics resampled
from the original topic set Q:

DISJOINT (i.e. original Voorhees/Buckley)
which samples topics WITHOUT replacement from Q to create the first topic set,
and then samples topics WITHOUT replacement FROM THE REMAINING TOPICS to 
create the second topic set, thus ensuring that the two sets are disjoint;

REPLACEMENT which samples topics WITH replacement from Q (just like the Bootstrap)
to create the two sets independently;

INDEPENDENT which samples topics WITHOUT replacement from Q 
to create the two sets independently.

In this paper, I described REPLACEMENT as a method which Sanderson and Zobel
studied in their SIGIR 2005 paper,
based on the fact that they referred to their own method as 
"selection WITH replacement" and the original Voorhees/Buckley method
as "selection WITHOUT replacement".

However, on reading this paper, Mark Sanderson contacted me and told me 
that what they in fact used was what I call INDEPENDENT, not REPLACEMENT.
That is to say, Sanderson and Zobel never used topic sampling WITH replacement.
 
Therefore, our experiments, which showed that 
DISJOINT and REPLACEMENT lead to similar results while INDEPENDENT gives 
lower swap rates, do not necessarily contradict with those 
reported by Sanderson and Zobel, who meant that INDEPENDENT represents 
swap rate lowerbounds.


Tetsuya Sakai
February 5, 2006.