Improvements in Holistic Recommender System Research

Abstract

Since the mid 1990s, recommender systems have grown to be a major area of deployment in industry, and research in academia. A through-line in this research has been the pursuit, above all else, of the perfect algorithm. With this admirable focus has come a neglect of the full scope of building, maintaining, and improving recommender systems. In this work I outline a system deployment and a series of offline and online experiments dedicated to improving our holistic understanding of recommender systems. This work explores the design, algorithms, early performance, and interfaces of recommender systems within the scope of how they are interconnected with other aspects of the system. This work explores many indivisual aspects of a recommender system while keeping in mind how they are connected to other aspects of the system. The contributions of this thesis are: an exploration of the design of the BookLens system, a prototype recommender system for library-item recommendation; a methodology and exploration of algorithm performance for users with very few ratings which shows that the popular Item-Item recommendation algorithm performs very poorly in this context; an exploration of the issues faced by Item-Item, as well as fixes for these issues confirmed by both an offline and online analysis; and finally, the preference bits model for measuring the amount of noise and information contained in user ratings, as well as a rating support interface capable of reducing the noise in user ratings leading to superior algorithm performance. Supporting these contributions are the following specific methodological improvements: a bias free methodology for measuring algorithm performance over a range of profile sizes; a prototype user-study design for investigating new-user recommendation through Amazon Mechanical Turk; the preference bits model as well as derived measurements of preference bits per rating, per impressions, and per second; and finally a sound experimental design that can be used to empirically measure preference bits values for a given interface. It is our hope that these methodological contributions can help researchers in the recommender systems field ask new questions and further the holistic study of recommender systems.