Web Mining: Information and Pattern Discovery on the World Wide Web

Abstract

Two important and active areas of current research are data mining and the World Wide Web. A natural combination of the two areas, sometimes referred to as Web mining, has been the focus of several recent research projects and papers. As with any emerging research area there is no established vocabulary, leading to confusion when comparing research efforts. Different terms for the same concept or different definitions being attached to the same word are commonplace. The term Web mining has been used in two distinct ways. The first, which is referred to as Web content mining in this paper, describes the process of information or resource discovery from millions of sources across the World Wide Web. The second, which we call Web usage mining, is the process of mining Web access logs or other user information user browsing and access patterns on one or more Web localities. In this paper we define Web mining and, in particular, present an overview of the various research issues, techniques, and development efforts in Web content mining and Web usage mining. We focus mainly on the problems and proposed techniques associated with Web usage mining as an emerging research area. We also present a general architecture for Web usage mining and briefly describe the WEBMINER, a system based on the proposed architecture. We conclude this paper by listing issues that need the attention of the research community.