Browsing by Subject "Dynamic treatment regimes"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Modifications of Q-learning to Optimize Dynamic Treatment Regimes
    (2021-08) Zhang, Yuan
    With an emerging interest in personalized medicine and quality healthcare, the design of clinical trials that incorporates multiple stages of randomization and intervention, for example, a sequential multiple assignment randomized trial (SMART), has become a popular choice for investigators as it facilitates the construction and analysis of dynamic treatment regimes (DTRs). There exists a comprehensive body of literature on various statistical methods to analyze data collected from such trials and estimate the optimal DTR for an individual subject, among which Q-learning with linear regression is widely used due to its simplicity and ease of interpretation. This thesis discusses three important challenges that cause problems in the implementation of Q-learning and proposes multiple modifications of Q-learning to address them.The first challenge arises from the repeatedly monitored outcome of interest at intermediate stages of randomization and at longer follow-up intervals after the final stage of randomization. Clinical investigators are usually interested in identifying the optimal DTR and estimating the outcome trajectory under the optimal DTR. However, in the presence of stagewise repeated-measures outcomes, standard Q-learning fails to provide point estimates of the optimal trajectory with time-specific heterogeneous causal effects. To address this problem, we propose a modified algorithm of Q-learning with a generalized estimating equation to estimate each Q-function. The second challenge is model misspecification. Model misspecification is a common problem in Q-learning, but little attention has been given to its impact when treatment effects are heterogeneous across subjects. We describe the integrative impact of two possible types of model misspecification related to treatment effect heterogeneity: unexplained early-stage treatment effects in late-stage main effect model, and misspecified linearity between pseudo-outcomes and predictors as a result of the optimization operation. The proposed method, aiming to deal with both types of misspecification concomitantly, builds interactive models into residual-modified parametric Q-learning. The third challenge is generalizing modified Q-learning to dichotomous outcomes. It is difficult to include informative residuals from estimation of late-stage models into early-stage pseudo-outcomes due to the non-identity link function. We propose a modification based on monotonicity of preferences to address model misspecification in Q-learning with probit regression. The improvement in robustness of the proposed modification is subject to the extent of model misspecification and can be limited. Thus, we take a latent variable approach and propose a novel algorithm using sampled surrogates of the underlying continuous outcome conditional on the binary observations. The methods proposed in this thesis are assessed via simulations and illustrated using the M-bridge study, a SMART with embedded tailoring which develops and evaluates adaptive interventions for preventing binge drinking among college students.
  • Thumbnail Image
    Item
    Policy-relevant causal effect estimators for heterogeneous treatments and exposures
    (2021-12) Lyden, Grace
    Most statistical methods for causal inference are designed to handle contrasts between well-defined treatment groups, e.g., vaccine versus placebo. In real-world applications, however, these contrasts might fail to answer relevant questions for patients and policy makers. This dissertation introduces new policy-relevant causal estimators that target the effects of heterogeneous treatments and exposures in observational data. Chapter 2 is motivated by correlated chemical mixtures. Historically, environmental health researchers have estimated separate effects of each chemical in a family. More recently, federal agencies have called for estimation of overall mixture effects, which acknowledge the potential real-world burden of simultaneous exposure. A secondary goal of mixtures research is to identify the most harmful components for regulation. Weighted Quantile Sum (WQS) regression has emerged to answer this call. WQS assigns a regularized weight to each chemical in a mixture through a form of bootstrap aggregation, then tests the effect of the weighted sum in a held-out dataset. Although popular, WQS is limited by its dependence on data splitting, which is needed to preserve Type I error. In Chapter 2, we propose the first modification of WQS that does not rely on data splitting and replaces the second step of WQS with a permutation test that yields correct p-values. To offset the added computational burden of a permutation test, we suggest alternatives to the bootstrap for regularization of the weights, namely L1 and L2 penalization, and discuss how to choose the appropriate penalty given expert knowledge about the mixture of interest. Chapters 3 and 4 are motivated by the difficult decisions faced by candidates for organ transplant. Due to organ scarcity, these patients typically have to wait to receive an offer of a suitable deceased-donor organ or possibly pursue living-donor transplant, depending on the organ needed. For patients who have that choice, the difference in post-transplant survival between living- and deceased-donor transplant is a straightforward quantity to estimate, but might not be particularly helpful to patients who experience real-world variability in wait time and offered-organ quality. A more useful contrast, therefore, is the survival difference between treatment strategies that account for this uncertainty, such as "wait for deceased-donor transplant," which could encompass many possible wait times and donor organ qualities. Decisions for patients today are further complicated if versions of treatment have changed over time, for example if the rate of transplant has changed due to evolutions in allocation policy. We, therefore, introduce the concept of a generalized representative intervention (GRI): a random dynamic treatment regime that assigns version of treatment with the distribution of similar patients in a target population under a loosely defined strategy of interest. Chapter 3 proposes a class of weighted product-limit estimators of marginal survival under a GRI, which are consistent, asymptotically normal, and have excellent finite-sample performance in a realistic simulation study. Chapter 4 extends this work to determine the optimal strategy for an individual based on their expected rate of treatment in the target population. Specifically, we propose a marginal structural modeling approach that allows a patient-specific relative rate of treatment to modify the effects of the GRIs under consideration. We apply our methods to data from the Scientific Registry of Transplant Recipients to determine the optimal strategy for kidney-pancreas transplant candidates under the current organ allocation system.