Adaptive learning and prediction: a deep-in-depth investigation into combining density forecast, Adaptive Regression by Mixing, and multi-armed bandits problems

Abstract

This dissertation investigates two key areas: model averaging and multi-armed bandits problems, both of which have seen significant research over the decades. Model averaging focuses on combining multiple models to create a more robust and even powerful method, while multi-armed bandits aim to identify the optimal strategy that maximizes “rewards” from various actions. To begin with, we explore a specialized generalized pooling scheme, the geometric pooling with logarithm monotone transformation, for combining density forecasts in univariate time series settings. We also introduce the L2 penalty in optimization for the best combin- ing weights to achieve the “grouping effect”. Some asymptotic behaviors of the optimized weights are investigated in the theoretical analysis. Those behaviors are also verified numerically through a comprehensive simulation study. Next we introduce a localized approach to the Adaptive Regression by Mixing (ARM) combining method in regression settings. Our model combining scheme calculates the variances and weights for candidate methods in a local manner, focusing on distinct blocks within a partition for the sample space of predictors. Three algorithms are designed to implement the localized ARM method in practice. Theoretically, we obtain an improved (sharper) version of the upper bound for ARM by leveraging information about the sample space, which demonstrates the advantages of employing localized ARM when no individual candidate method consistently performs well across all locations. Two comprehensive simulation studies are provided, offering illustration and numerical verification for our proposed techniques and theorems. Finally, in the context of contextual multi-armed bandit problems, we consider three types of generalized rewards that account for error variability. We propose algorithms, or allocation rules, for pulling arms using annealed ε-greedy randomized allocation to balance the exploration and exploitation processes. For the estimation of the mean reward, we focus on the histogram method as a nonparametric regression procedure. Both the theoretical analysis and simulation studies suggest that, under some mild conditions, the per-round mean regret has strong consistency with our allocation rules.