Browsing by Subject "multi-task learning"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Advanced learning approaches based on SVM+ methodology.
    (2011-07) Cai, Feng
    Exploiting additional information to improve traditional inductive learning is an active research area in machine learning. In many supervised learning applications, training data contains additional information not reflected in training pairs . Examples include: (1) time series prediction where future samples can be observed in the training data, (2) handwritten digit recognition where training examples are provided by several persons, and this group information is not utilized during training, (3) medical diagnosis where predictive (diagnostic) model, say for lung cancer, is estimated using a training set of male and female patients. The gender can be considered as additional group information. Incorporating this additional information into learning may improve generalization. Recently, Vapnik proposed a general approach for incorporating additional information into learning, known as Learning Using Privileged Information (LUPI) and learning with structured data (LWSD) which utilizes group information (Vapnik, 2006). A SVM based methodology SVM+ was proposed under LUPI and LWSD setting (Vapnik, 2006). In this thesis, we will first introduce a SVM+ based feature selection system. Then we extend SVM+ to multi-task learning (MTL) setting, where both training and test data can be naturally partitioned into several groups. SVM+ based MTL (SVM+MTL) method for both classification and regression are proposed and analyzed. SVM+MTL estimates multiple models simultaneously, i.e. one model for each group/task. Task inter-dependency is modeled by sharing a common part of the decision function among different groups. Connections and differences between SVM+ and SVM+MTL are discussed. Practical parameter tuning strategies are proposed for SVM+MTL. Empirical comparisons show that SVM+MTL works very well on data sets with group information. Finally, generalized sequential minimal optimization (GSMO) methods are proposed for SVM+MTL training, for both classification and regression settings.
  • Thumbnail Image
    Item
    Predictive Learning with Heterogeneity in Populations
    (2017-10) Karpatne, Anuj
    Predictive learning forms the backbone of several data-driven systems powering scientific as well as commercial applications, e.g., filtering spam messages, detecting faces in images, forecasting health risks, and mapping ecological resources. However, one of the major challenges in applying standard predictive learning methods in real-world applications is the heterogeneity in populations of data instances, i.e., different groups (or populations) of data instances show different nature of predictive relationships. For example, different populations of human subjects may show different risks for a disease even if they have similar diagnosis reports, depending on their ethnic profiles, medical history, and lifestyle choices. In the presence of population heterogeneity, a central challenge is that the training data comprises of instances belonging from multiple populations, and the instances in the test set may be from a different population than that of the training instances. This limits the effectiveness of standard predictive learning frameworks that are based on the assumption that the instances are independent and identically distributed (i.i.d), which are ideally true only in simplistic settings. This thesis introduces several ways of learning predictive models with heterogeneity in populations, by incorporating information about the context of every data instance, which is available in varying types and formats in different application settings. It introduces a novel multi-task learning framework for problems where we have access to some ancillary variables that can be grouped to produce homogeneous partitions of data instances, thus addressing the heterogeneity in populations. This thesis also introduces a novel strategy for constructing mode-specific ensembles in binary classification settings, where each class shows multi-modal distribution due to the heterogeneity in their populations. When the context of data instances is implicitly defined such that the test data is known to comprise of contextually similar groups, this thesis presents a novel framework for adapting classification decisions using the group-level properties of test instances. This thesis also builds the foundations of a novel paradigm of scientific discovery, termed as theory-guided data science, that seeks to explore the full potential of data science methods but without ignoring the treasure of knowledge contained in scientific theories and principles.