A Novel Regression Model Combining Instance Based Rule Mining With EM Algorithm

Abstract

In recent years, there have been increasing efforts to apply association rule mining to build Associative Classification (AC) models. However, the similar area that applies association rule mining to build Associative Regression (AR) models has not been well explored. In this work, we fill this gap by presenting a novel regression model based on association rules called AREM. AREM derives a set of regression rules by: (i) applying an instance based approach to mine itemsets which form the regression rules' left hand side, and (ii) developing a probabilistic model which determines, for each mined itemset, the corresponding rule's right hand side and the importance weight. To address the computational bottleneck of the traditional two-step approach for itemset mining, AREM utilizes an Instance-Based Itemset Miner (IBIMiner) algorithm that directly discovers the final set of itemsets. IBIMiner incorporates various methods to bound the quality of any future extensions of the itemset under consideration. These bounds are then used to prune the search space. In addition, AREM treats the regression rules' right hand side and importance weights as parameters of a probabilistic model, which are then learned in the expectation and maximization (EM) framework. The extensive experimental evaluation shows that our bounding strategies allow IBIMiner to considerably reduce the runtime and the EM optimization can improve the predictive performance dramatically. We also show that our model can perform better than some of the state of the art regression models.