Understanding Categorical Similarity Measures for Outlier Detection

Abstract

Categorical attributes are present is many data sets that are analyzed using KDD techniques. A recent empirical study of 14 different data driven categorical similarity measures in the context of outlier detection showed that these measures have widely different performances when applied to several different publicly available data sets. As a next step in understanding the relation between the performance of a similarity measure and the nature of the data, we present an analysis framework to help give insights as to which similarity measure is better suited for what type of data. In this paper we present a framework for modeling categorical data sets with a desired set of characteristics. We also propose a set of separability statistics for a categorical data set that can be used to understand the performance of a similarity measure for outlier detection. In addition, we present three techniques to estimate the proposed separability statistics from a given categorical data set. We experimentally evaluate the different similarity measures in the context of outlier detection, and show how the performance of a similarity measure is related to the various data characteristics.