Significance testing for INEX 2008-09 Ad Hoc Track
2010-08
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
Significance testing for INEX 2008-09 Ad Hoc Track
Authors
Published Date
2010-08
Publisher
Type
Thesis or Dissertation
Abstract
INEX (The INitiative for the Evaluation of XML retrieval) sponsors a competition
that promotes the development/evaluation of XML-based retrieval systems. INEX
provides a document collection, a query set (topics) and evaluation measures for use by
the XML-based retrieval systems of the participants. We have developed a methodology
for the retrieval of elements, at the appropriate level of granularity, within the XML
document. This methodology is applied to the tasks of the INEX Ad Hoc Track.
In this thesis, the focus lies on significance tests that are performed to see how our
methods compare with those used by the top-ranked INEX participants. Our approach to
significance testing is identical to that used by INEX for evaluating its participants. We
use the results of each individual query; find the variance and standard deviation between
the scores, and the value of t, which maps to probability. We use a confidence interval of
95% in one-tailed t-test (so the probability must be less than 0.05 to assure significance)
to see if one method is significantly better than another. We evaluate results of the 2008
and 2009 Ad Hoc tasks using this approach. The results of these significance tests for the
basic INEX Ad Hoc tasks are given in this paper along with observations on these results.
Description
University of Minnesota M.S. thesis. August 2010. Major: Computer science: Advisor: Dr. Carolyn J. Crouch. 1 computer file (PDF); vi, 47 pages, appendix A.
Related to
Replaces
License
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Suggested citation
Cherukuri, Ramakrishna. (2010). Significance testing for INEX 2008-09 Ad Hoc Track. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/101638.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.