Advancing regression and causal inference: invariance, shape constraints, and multivariate dependence

Abstract

Modern statistical analysis faces mounting challenges driven by the increasing complexity of data, structural dependencies, and evolving scientific questions.As the demand grows for methods that balance theoretical rigor, computational efficiency, and practical relevance, the need to rethink classical modeling and inference approaches becomes more apparent. This dissertation presents a collection of works that address fundamental statistical problems arising in regression and causal inference, each motivated by real and persistent gaps in the existing methodologies. The first line of work investigates the issue of invariance in linear regression, where widely used penalized regression methods often yield fitted values sensitive to invertible linear transformations of the model matrix. To address this concern regarding stability and reliability, we propose a shrinkage method that aims to ensure fitted value invariance under invertible linear transformations of the model matrix. The second part of this dissertation focuses on distributional comparisons in causal inference. While average treatment effects provide a convenient summary of mean-level differences, they often fail to reveal subtle and scientifically meaningful differences across the distributions of potential outcomes. To address this issue, we develop a shape-constrained estimation and inference framework based on log-concavity, which enables asymptotically valid honest inference for counterfactual densities while requiring minimal tuning and circumventing the difficulties of traditional nonparametric methods. The third and final part of this dissertation investigates multivariate regression when responses are strongly correlated through dense error structures. While existing methods typically rely on sparsity assumptions for the error precision matrix, such approaches may struggle in these settings. To address this challenge, we propose a regression framework that leverages compound symmetry (or equicorrelation) error structures, providing a parsimonious yet effective solution that enables efficient estimation and remains robust under moderate error covariance misspecification. Together, these contributions advance the development of regression and nonparametric causal inference methodologies that are theoretically solid, computationally viable, and practically attuned to the realities of modern statistical applications.