Reading Between the Tweets: Clustering and Topic Extraction on Micro-Blogs

Abstract

According to Java et al., some of the main intentions of users on Twitter are daily chatting, conversations, sharing information/URLs, and reporting news. Topic extraction has been used in large documents such as online news articles, emails, scientific literature, and blogs, but little has been done with short documents like micro-blogs. Objective: Apply clustering and topic extraction techniques to micro-blogs as a first step toward tracking popular topics dynamically as they vary by time or location, or to enhance topic-related searches of tweets. The results found using a hybrid PDDP and K-Means clustering technique, typically used for categorizing large documents, have shown success when working with micro-blogs with noisy dictionaries. The methods show potential to be a basis for [semi-]supervised topic tracking and extraction using methods previously used for larger text documents.