Spectral curvature clustering for hybrid linear modeling.

Abstract

The problem of Hybrid Linear Modeling (HLM) is to model and segment data using a mixture of affine subspaces. Many algorithms have been proposed to solve this problem, however, probabilistic analysis of their performance is missing. In this thesis we develop the Spectral Curvature Clustering (SCC) algorithm as a combination of Govindu's multi-way spectral clustering framework (CVPR 2005) and Ng et al.'s spectral clustering algorithm (NIPS 2001) while introducing a new affinity measure. Our analysis shows that if the given data is sampled from a mixture of distributions concentrated around affine subspaces, then with high sampling probability the SCC algorithm segments well the different underlying clusters. The goodness of clustering depends on the within-cluster errors, the between-clusters interaction, and a tuning parameter applied by SCC. Supported by the theory, we then present several novel techniques for improving the performance of the algorithm. Specifically, we suggest an iterative sampling procedure to improve the existing uniform sampling strategy, an automatic scheme of inferring the tuning parameter from data, a precise initialization procedure for K-means, as well as a simple strategy for isolating outliers. The resulting algorithm requires only linear storage and takes linear running time in the size of the data. We compare it with other state-of-the-art methods on a few artificial instances of affine subspaces. Application of the algorithm to several real-world problems is also discussed.