Browsing by Author "Uygar Oztekin, B."

Now showing 1 - 5 of 5
  • Thumbnail Image
    Item
    Average Position of User Clicks as an Automated and Non-Intrusive Way of Evaluating Ranking Methods
    (2003-02-05) Uygar Oztekin, B.; Karypis, George; Kumar, Vipin
    The need for an objective and automated way of evaluating the performance of different ranking/reranking methods is becoming increasingly important in the web search/meta search domain. There are various methods for ranking search results ranging from traditional information retrieval approaches to more recent methods based on link analysis and other quality measures that can be derived from the documents. There are also a number of strategies for combining different heuristics and answers from multiple experts. With all of these possibilities it is becoming increasingly difficult to find the best parameters, the best method, or the best mixture of methods that will maximize the quality for a particular query type or domain. This paper addresses the problem of automatically comparing the quality of the ordering of documents that are presented to the user as a sorted list according to believed relevance for a given topic or query. We introduce the average position of user clicks metric as an implicit, automated, and non-intrusive way of evaluating ranking methods. We also discuss under which situations and assumptions this metric can be used objectively by addressing various bias sources. Experiments performed in our meta search engine suggests that, this approach has the potential to sample a wide range of query types and users with greater statistical significance compared to methods that rely on explicit user judgements.
  • Thumbnail Image
    Item
    Expert Agreement and Content Based Reranking in a Meta Search Environment Using Mearf
    (2001-07-16) Uygar Oztekin, B.; Karypis, George; Kumar, Vipin
    Recent increase in the number of search engines on the Web and the availability of meta search engines that can query multiple search engines makes it important to find effective methods for combining results coming from different sources. In this paper we introduce novel methods for reranking in a meta search environment based on expert agreement and contents of the snippets. We also introduce an objective way of evaluating different methods for ranking search results. Our experimental evaluation shows that some of our methods produce rankings that are consistently better than the rankings produced by methods that are commonly used in many meta search engines as well as rankings produced by a popular search engine.
  • Thumbnail Image
    Item
    Personalized Profile Based Search Interface With Ranked and Clustered Display
    (2001-06-01) Kumar, Sachin; Uygar Oztekin, B.; Ertoz, Levent; Singhal, Saurabh; Han, Euihong; Kumar, Vipin
    We have developed an experimental meta-search engine, which takes the snippets from traditional search engines and presents them to the user either in the form of clusters, indices or re-ranked list optionally based on the user’s profile. The system also allows the user to give positive or negative feedback on the documents, clusters and indices. The architecture allows different algorithms for each of the features to be plugged-in easily, i.e. various clustering, indexing and relevance feedback algorithms, and profiling methods.
  • Thumbnail Image
    Item
    Usage Aware PageRank
    (2003-02-05) Uygar Oztekin, B.; Ertoz, Levent; Kumar, Vipin; Srivastava, Jaideep
    Traditional link analysis approaches assume equal weights assigned to different links and pages. In original PageRank formulation, the user model assumes that the user has equal probability to follow each link from a given page, thus the score of a page equally affects all of the pages it points to. It also assumes that the probability for a user to go to a URL directly without following a link is the same for all URLs. In this paper, we investigate different weighting schemes that take into account the probability to go to a page directly (by typing or using bookmarks), as well as the relative probability to follow a link from a given page. Both of these probabilities can be approximated from usage logs if they are available. We introduce a naturalextension to the original PageRank formulation that we will call Usage aware PageRank (UPR). The new formulation combines static link structure graph with the usage graph that will be obtained via web logs or other means. It is also quite general; how much emphasis will be given to the graphs is controlled by a parameter. If the parameter is set to zero, the algorithm becomes equivalent to the original PageRank, if it is set to one, the emphasis shifts to the usage graph, and for values in between, both of the graphs will be used with weights specified by the parameter. UPR is also quite inexpensive. After a onetime precalculation step, an iteration of UPR takes about the same time as a PageRank iteration.
  • Thumbnail Image
    Item
    Usage Meets Link Analysis: Towards Improving Site Specific and Intranet Search via Usage Statistics
    (2004-05-24) Uygar Oztekin, B.; Kumar, Vipin
    In this paper, we explore the possibility of incorporating usage statistics to improve ranking quality in site specific and intranet search engines. We introduce a number of usage based ranking approaches including a PageRank extension, Usage aware PageRank (UPR), an extension to HITS (UHITS), and a naive approach that uses number of visits to pages as a quality measure. We compare these methods against each other and against two major link analysis approaches (PageRank and HITS). We investigate weighting schemes that take into account the probability of visiting a page directly (by typing or via bookmarks), as well as the relative probability of following a particular link from a given page. Both of these probabilities can be approximated from usage logs. We developed a site specific search engine (http://usearch.cs.umn.edu/), and incorporated the above methods. The parameter space for UPR and UHITS are sampled to examine the effects of varying usage emphasis factors. Experimental results are carried out on a medium size domain, cs.umn.edu, with 20K static web pages. We provide both global and query dependent comparisons. Experiments suggest that UPR is promising and has a number of desirable properties. It generalizes PageRank and inherits basic PageRank properties. It is also stable and flexible. The emphasis given to usage information is controlled via two parameters. If the parameters are set to zero, the algorithm reduces to the original PageRank algorithm; if they are set to one, the emphasis shifts to the usage graph; for values in between, both of the graphs are used with the specified weights. UPR is relatively inexpensive. The usage graph can be updated incrementally and efficiently as new usage information becomes available. A UPR iteration has a space/time complexity similar to a PageRank iteration.