Browsing by Subject "optimization"

Now showing 1 - 16 of 16

Advancing Cell Culture Engineering Through Mechanistic Model Optimization
(2020-04) O'Brien, Conor
Over the past few decades, the emergence of new classes of treatments, including protein therapeutics, gene therapies, and cell therapies, has ushered in a new era of medicine. Unlike small molecule therapeutics, these treatments are produced in or consist of cells, typically mammalian in origin. Processes have been developed to produce many of these drugs at large scale, often in stirred tank bioreactors. Significant effort has driven staggering increases in the productivity of these processes, enabling economical manufacturing, and the potential to drive down costs and make drugs more widely available. However, the bioreactor is not a natural environment for cells isolated from a multicellular mammalian organism. Many biological regulations are carried over from the cells’ origin and can result in numerous undesirable behaviors manifesting in the dense, highly productive reactor environment. In certain culture stages, or in the case of excess nutrient supply, cells will secrete undesirable metabolites including lactate, ammonia, and many byproducts of amino acid metabolism. These compounds can retard cell growth, or otherwise alter the potency or productivity of the cultures. Traditional biologics process development employs the use of statistical design of experiments, often encompassing several reactors run in parallel for multiple rounds of experiments over a few months. There is thus substantial room for improvement for both the outcome of the development process, such as an increase in titer, and the time it takes to complete the development stage. Given that cell culture processes share intrinsic similarities in their underlying mechanistic behavior, there exists significant opportunity to reduce the overall number of experiments needed for process development, scaling, and diagnostics using models rather than treating cell culture processes as a black box. In this thesis, we present the case for the use of mathematical optimization of mechanistic models to accurately describe cell culture processes and augment their behavior. We first outline recent advances in understanding of metabolic regulation and homeostasis. Cell signaling and metabolic networks interact over multiple time-scales and through multiple means, resulting in cell metabolism with nonlinear behavior that is consequently context-dependent. In the following sections of this work, we then develop an optimization framework which can efficiently be used for the design of experiments to rewire cellular metabolism through metabolic engineering, or to otherwise understand the biological requirements of different metabolic phenomena. This framework is first applied to the Warburg effect, a century-old unsolved problem of rapid lactate production in proliferating cells to identify which enzymes may be altered to mitigate the lactate production. This framework in then applied to the problem of hepatic gluconeogenesis to study metabolic disease. As the expression of the enzymes specific to gluconeogenesis is not sufficient for glucose production, we explore what other requirements exist for the synthesis of glucose from different substrates. The next portion discusses the construction and optimization of a bioprocess model which includes metabolism, signaling, cell growth, and the reactor environment. This model is fit to a manufacturing-scale dataset to explore the origins of process variability and potential mitigation strategies. In the final segment of this thesis, we explore another aspect of protein therapeutics: product quality. A model of N-glycosylation is optimized in conjunction with successive rounds of experimentation with the goal of improving the galactose content on an antibody. This work highlights the benefits of feeding back experimental data to refine model parameters for better design and prediction of subsequent experiments.
Computational construction and simulation of novel heart valve and vein valve designs
(2023-04) Li, Jirong
As a heart valve substitute with growth potential and improved durability, tissue-engineered heart valves can prevent reinterventions that are currently often needed in children with congenital heart defects. Our group successfully made a pediatric tri-tube/tri-leaflet valved conduit which has shown growth capability in growing lambs. However, the optimal design for valve performance is unknown. To obtain optimal values of valve geometry parameters which can provide efficient guidance for animal tests to save costs and time, we utilized computer-aided simulations to evaluate multiple valve designs. We performed both the valve construction process and this optimization in silico. This is a complex optimization problem due to multiple components of the objective function for valve performance. We thus applied a multi-objective genetic algorithm, which is an elitist strategy, a parameter-free, simple, yet efficient, constraint-handling method used in many applications. A robust finite element method (FEM)-based algorithm for in silico construction of the valve was developed to facilitate optimization for the case of valve closure, identifying the optimal leaflet height and tube diameter that minimizes peak diastolic stress and maximizes coaptation. A strain energy constitutive equation for our novel material, an aligned tissue grown in the lab, was developed based on biaxial testing and implemented in the FEM model. Fluid-structure interaction (FSI) simulation of the optimal design under steady flow was also conducted as a prelude to a next-stage optimization that includes valve dynamic performance metrics. The same methods for the heart valve were also applied to the computational construction and simulation of a stented bileaflet venous valve that includes a sinus design. Steady FSI simulations were used to investigate the effects of the sinus geometry and non-Newtonian blood rheology on the hemodynamics of the novel transcatheter venous valve.
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
(2022-03) Chen, Xiangyi
Recently, there is a growing interest in the study of median-based algorithms for distributed non-convex optimization. Two prominent examples include signSGD with majority vote, an effective approach for communication reduction via 1-bit compression on the local gradients, and medianSGD, an algorithm recently proposed to ensure robustness against Byzantine workers. The convergence analyses for these algorithms critically rely on the assumption that all the distributed data are drawn iid from the same distribution. However, in applications such as Federated Learning, the data across different nodes or machines can be inherently heterogeneous, which violates such an iid assumption. This work analyzes signSGD and medianSGD in distributed settings with heterogeneous data. We show that these algorithms are non-convergent whenever there is some disparity between the expected median and mean over the local gradients. To overcome this gap, we provide a novel gradient correction mechanism that perturbs the local gradients with noise, which we show can provably close the gap between mean and median of the gradients. The proposed methods largely preserve nice properties of these median-based algorithms, such as the low per-iteration communication complexity of signSGD, and further enjoy global convergence to stationary solutions. Our perturbation technique can be of independent interest when one wishes to estimate mean through a median estimator.
Formulation and Analysis of an Optimization-Based Atomistic-to-Continuum Coupling Algorithm
(2015-09) Olson, Derek
Atomistic-to-continuum (AtC) methods are multiphysics models of materials used to simulate atomistic systems on a size scale unreachable on even the largest modern computers. A cornucopia of AtC algorithms has been designed and implemented, but the core feature among them is the decomposition of the computational domain into an atomistic region modeled using an atomistic model and a continuum region modeled according to some continuum mechanics formulation such as nonlinear elasticity. Though much information can be gleaned from the individual atomistic and continuum models, the mathematical analysis of the errors involved in the coupled approximations are just now beginning to be understood. Such an analysis is vital to both guide the choice of AtC method and optimize its efficiency. A convenient model problem for comparing the errors of various AtC methods has been simulating a single defect embedded in an infinite crystalline environment in Euclidean space [36,42]. This allows the error to be generically decomposed into a far-field error resulting from truncating the problem to an infinite domain, a coarsening error due to efficiently solving the continuum problem (e.g. using finite elements), and a coupling error arising from coupling the two models. This work presents the optimization-based AtC algorithm of [48] in the context of a point defect in an infinite lattice and provides error estimates for the three aforementioned errors in two and three dimensions. The method relies on overlapping atomistic and continuum regions with individual atomistic and continuum subproblems. These two problems are coupled together using a virtual control technique originally developed in [26] to couple advection-diffusion PDEs. A cost functional consisting of the H1 seminorm of the difference between atomistic and continuum states is minimized subject to the constraints that the atomistic and continuum equilibrium equations hold on each subdomain.
GIS-Based Comprehensive Shared Micromobility Station Siting Optimization for Small Urban Areas
(2023) DeBruin, Hannah
Shared micromobility travel modes such as dockless e-scooters and bikeshare programs have become increasingly popular in the United States in the last decade because of their potential to improve multi-modal accessibility within communities. Smaller urbanized areas with lower population densities and fewer resources for system planning, operation, and maintenance present unique challenges with siting optimal station/service area locations. This research develops a comprehensive GIS-based methodology for optimizing micromobility stations/service area locations using available and rasterized geospatial data to capture bikeshare demand indicators. Inputs are prioritized by overall importance according to the results of a survey of transportation professionals, with weights calculated by an analytic hierarchy process. These different factors are combined to create a new spatial index value to identify hotspots for candidate station/service area locations, which can be further analyzed to choose optimal locations based on the budgeted quantity of station/service area locations and ideal spacing between stations/service areas. The case study of the methodology is presented on a bikeshare station siting study in Iowa City, Iowa using data from the Metropolitan Planning Organization of Johnson County. This research seeks to improve shared micromobility station/service area planning to better orient service to a variety of transportation goals including regular/commuting use, recreational use, equity, first/last mile connection to transit, and operational partnership opportunities. Multimodal travel times and job accessibility in the study area are evaluated both before and after the introduction of bikeshare, and both greatly improve with the introduction of optimal stations. Public agencies could expect to benefit from this comprehensive methodology because it uses easily obtainable data sources and provides the flexibility to weight the importance of factors differently in order to fit their communities’ specific transportation goals.
Improving Virtual Impactor Performance via Nozzle Optimization
(2023-05) Relling, Mckenna
Virtual impactors are aerosol-concentrating devices composed of a nozzle and downstream receiving tube. The majority aerosol flow accelerated through the nozzle is diverted from transmitting through the receiving tube, yet particles inertially maintain trajectory into the tube, thereby increasing their concentrations. This study investigated the effect of nozzle geometry on particle focusing, a phenomenon wherein particles not only inertially enter the receiving tube but are radially confined to center streamlines. Specifically, the study aimed to understand how modifications to the nozzle's geometry can improve particle focusing. For this purpose, around 300 nozzles were simulated, with both fluid flow and particle trajectories modeled. The extent of focusing was quantified via a score metric determined by the fraction of particles confined to the inner 10% of the nozzle exit. The study found that shorter nozzles and simple geometries, such as those with a ratio of inlet radius to radius of curvature of around two, tend to have higher scores for particle focusing. The study also suggested the development of a two-stage nozzle to focus larger and smaller particles separately but concluded that the two stages cannot be selected independently of each other. It is suggested that the addition of a straight section between the two stages may allow for independent selection. Two focusing-optimized virtual impactor geometries were experimentally tested and compared to a reference geometry not optimized for focusing. All three impactor geometries were also simulated, and their penetration curves were compared to experimental data. Experimental data points were found to match well with their simulated curves, indicating accurate predictions of impactor performance. One of the optimized geometries showed slightly worse performance at small particle diameters, but no overfocusing at larger particle diameters, when compared to the reference geometry. The other optimized geometry showed similar performance at small particle diameters, but significantly improved penetration at larger particle diameters. Comparing the overfocusing via the receiving tube collection efficiencies, the two optimized virtual impactors showed 60% and 80% reductions in receiving tube losses, indicating improved particle focusing. The study concludes that the trajectory of particles can be controlled solely via the modification of nozzle geometry, which can be used in conjunction with a virtual impactor to improve the overall ability of the impactor to concentrate particles.
Matrix Completion via Nonconvex Factorization: Algorithms and Theory
(2015-05) Sun, Ruoyu
Learning low-rank structure of the data matrix is a powerful method to deal with ``big data''. However, in many modern applications such as recommendation systems and sensor localization, it is impossible or too costly to obtain all data, resulting in a data matrix with most entries missing. A problem of great interest is then to infer the missing data based on very few observations, usually under the assumption that the true data matrix is low rank, which is widely known as the matrix completion problem. The most popular solution for large-scale matrix completion is the matrix factorization (MF) formulation,which has achieved great empirical success. However, due to the non-convexity caused by the factorization model, little is known about whether and when the algorithms for the MF formulation will generate a good solution. In this thesis, we will study the non convex MF formulation for matrix completion, both algorithmically and theoretically. First, we empirically analyze several standard algorithms for the MF formulation. We present a novel finding that the recovery ability of an algorithm mainly depends on whether it can control the row-norms (or incoherence constants) of the iterates. Motivated by this finding, we propose a new formulation that either adds constraints or penalty-function-type regularizers to directly control the row-norms. Simulation results show that the algorithms for the new formulation can recover the matrix in the regime very close to the fundamental limit, outperforming the standard algorithms. We then establish a theoretical guarantee for the new MF formulation to correctly recover the underlying low-rank matrix. In particular, we show that under similar conditions to those in previous works, many standard optimization algorithms converge to the global optima of the new MF formulation, and recover the true low-rank matrix. This result is rather surprising since we prove convergence to global optima for a nonconvex problem. To the best of our knowledge, our result is the first one that provides exact recovery guarantee for many standard algorithms. A major difference of our work from the existing results is that we do not need resampling (i.e., using independent samples at each iteration) in either the algorithm or its analysis. Technically, we develop a novel perturbation analysis for matrix factorization which may be of independent interest.
Momentum for the Frank Wolfe Method
(2022-05) Li, Bingcong
Modern machine learning tasks built to learn from data can be typically formulated as optimization problems. The large volume of data justifies the pressing need for efficient and scalable iterative algorithms that are designed specifically to accommodate to the computation resource at hand and the requirement of structural (e.g., sparse) solutions. Conditional gradient, aka Frank Wolfe (FW) algorithms, have well-documented merits in machine learning and signal processing applications that involves minimizing a loss function with constraints. Compared to projection based methods, one of the key benefits is that FW overcomes the need of projection, which is computationally heavy. Unlike projection-based methods however, momentum cannot improve the convergence rate of FW, in general. For this reason, momentum is relatively less studied in the FW literature. This limitation motivates the work in this dissertation. In Chapter 2, we deal with heavy ball momentum and its impact to FW. Specifically, it is established that heavy ball offers a unifying perspective on the primal-dual (PD) convergence, and enjoys a tighter \textit{per iteration} PD error rate, for multiple choices of step sizes, where PD error can serve as the stopping criterion in practice. In addition, it is asserted that restart, a scheme typically employed jointly with Nesterov's momentum, can further tighten this PD error bound. Going beyond heavy ball momentum, we establish the connections between the subproblem in FW and Nesterov's momentum in Chapter 3. On the negative side, these connections show why momentum is unlikely to be effective for FW type algorithms on general problems. The encouraging message behind this link, on the other hand, is that Nesterov's momentum accelerates FW on a class of problems encountered in many signal processing and machine learning applications. In particular, we prove that a momentum variant of FW, that we term accelerated Frank Wolfe (AFW), converges with a faster rate ${\cal O}(\frac{1}{k^2})$ on such a family of problems despite the same ${\cal O}(\frac{1}{k})$ rate as FW on general cases. Our faster rates rely on parameter-free step sizes, which distinguishes with most of existing faster rates of FW variants. Chapter 4 introduces and analyzes a variant of FW termed ExtraFW. The distinct feature of ExtraFW is the pair of gradients leveraged per iteration, thanks to which the decision variable is updated in a prediction-correction (PC) format. Relying on no problem dependent parameters in the step sizes, ExtraFW convergences at a faster rate ${\cal O}\big(\frac{1}{k^2} \big)$ on a class of machine learning problems. Compared with other parameter-free FW variants that have faster rates on the same problems such as AFW, ExtraFW has improved rates and fine-grained analysis thanks to its PC update. Numerical tests on binary classification with different sparsity-promoting constraints demonstrate that the empirical performance of HFW, AFW and ExtraFW is significantly better than FW. We also observe that AFW and ExtraFW are even faster than Nesterov's accelerated gradient on certain datasets, even though they rely on no problem dependent parameters. For matrix completion, the solutions found by HFW, AFW and ExtraFW enjoy smaller optimality gap, and lower rank than FW.
Numerical Modeling And Optimization Of Thermofluid Systems: Heat Pumps, Turbocompressors, Porous Media
(2020-03) Goldenberg, Vlad
In this dissertation, three types of thermofluid systems: an air Brayton cycle heat pump, a centrifugal compressor stage, and a porous media heat pipe, are investigated. In each of the investigations, numerical modeling is used as the basis that underpins the analyses. Furthermore, the goal of each investigation is to develop a framework for the design and optimization of practical engineering systems. The parameterization of each system is explored and defined. A thermodynamic model of a recuperated air cycle heat pump is developed and used to parametrically study the effects of component performance, operating environment, and design parameters. A numerical optimization is conducted to maximize the heating COP of the air cycle heat pump while maintaining robust performance across a wide operating envelope. Comparison is made to a conventional vapor compression cycle heat pump. It is found that a judicious choice of pressure ratio and maximization of component performance enables a recuperated air cycle heat pump to be comparable in COP to a vapor cycle heat pump for high temperature ratio duty. The recommended pressure ratio is determined to be 1.4. Such a heat pump requires high performance compressor, expander, and heat exchangers. A novel method of the flow path synthesis of a centrifugal compressor stage is revealed. A preliminary design procedure that enables fast and efficient candidate designs is reported. Computational fluid dynamics in conjunction with optimization algorithms, surrogate modeling, and machine learning is used to analyze the fundamental fluid mechanics and to automatically optimize the designs. A single stage performance improvement of over 4% points of isentropic efficiency gain is demonstrated using numerical methods. The microstructure of a flat porous media heat pipe consisting of layers of wire mesh is characterized using numerical techniques. The analysis encompasses the characterizations of the flow-induced pressure drop and interfacial heat transfer for liquid and vapor water phases in a 16-gauge and 200-gauge wire mesh porous domain.
On Some Geometric Optimization Problems in Layered Manufacturing
(1997) Majhi, J.; Janardan, R.; Smid, M.; Gupta, P.
Efficient geometric algorithms are given for optimization problems arising in layered manufacturing, where a 3D object is built by slicing its CAD model into layers and manufacturing the layers successively. The problems considered include minimi7,ing the degree of stair-stepping on the surfaces of the manufactured object, minimizing the volume of the so-called support structures used, and minimizing the contact area between the supports and the manufactured object- all of which are factors that affect the speed and accuracy of the process. The stair-step minimization algorithm is valid for any polyhedron, while the support minimization algorithms are applicable t.o convex polyhedra only. Algorithms arc a1so given for optimizing supports for non-convex, simple polygons. The techniques used to obtain these results include construction and searching of certain arrangements on the sphere, 3D convex hulls, balfplanc range searching, ray-shooting, visibility, and constrained optimization.
On Variance Reduction in Machine Learning
(2021-04) Li, Bingcong
The quick evolution and widespread applicability of machine learning and artificial intelligence have fundamentally shaped and transcended modern life. Modern machine learning tasks built to learn from data can be typically formulated as empirical risk minimization (ERM) problems. The large amount of data justifies the pressing need for efficient and scalable optimization algorithms that are designed specifically for learning with big data. The main theme of Chapter 2 is a unifying algorithm, \textbf{L}oop\textbf{L}ess \textbf{S}ARAH (L2S) for ERM problems formulated as summation of $n$ individual loss functions. L2S broadens a recently developed variance reduction method known as SARAH. To find an $\epsilon$-accurate solution, L2S enjoys a complexity of ${\cal O}\big( (n+\kappa) \ln (1/\epsilon)\big)$ for strongly convex problems. For convex problems, when adopting an $n$-dependent step size, the complexity of L2S is ${\cal O}(n+ \sqrt{n}/\epsilon)$; while for more frequently adopted $n$-independent step size, the complexity is ${\cal O}(n+ n/\epsilon)$. Distinct from SARAH, our theoretical findings support an $n$-independent step size in convex problems without extra assumptions. For nonconvex problems, the complexity of L2S is ${\cal O}(n+ \sqrt{n}/\epsilon)$. Our numerical tests on neural networks suggest that L2S can have better generalization properties than SARAH. Along with L2S, our side results include the linear convergence of the last iteration for SARAH in strongly convex problems. However, variance reduced algorithms typically require grid search to tune parameters (step size and the number of iterations per inner loop) for optimal performance. To overcome this issue, Chapter 3 introduces `almost tune-free' SVRG and SARAH schemes equipped with i) Barzilai-Borwein (BB) step sizes; ii) averaging; and, iii) the inner loop length adjusted to the BB step sizes. In particular, SVRG, SARAH, and their BB variants are first reexamined through an `estimate sequence' lens to enable new averaging methods that tighten their convergence rates theoretically, and improve their performance empirically when the step size or the inner loop length is chosen large. Then a simple yet effective means to adjust the number of iterations per inner loop is developed to enhance the merits of the proposed averaging schemes and BB step sizes. Numerical tests corroborate the proposed methods. The main goal of Chapter 4 is equipping convex and nonconvex problems with Barzilai-Borwein (BB) step size. With the adaptivity of BB step sizes granted, they can fail when the objective function is not strongly convex. To overcome this challenge, the key idea here is to bridge (non)convex problems and strongly convex ones via regularization. The proposed regularization schemes are \textit{simple} yet effective. Wedding the BB step size with a variance reduction method, e.g., SARAH, offers a free lunch compared with vanilla SARAH in convex problems. The convergence of BB step sizes in nonconvex problems is also established and its complexity is no worse than other adaptive step sizes such as AdaGrad. As a byproduct, our regularized SARAH methods for convex functions ensure that the complexity to find $\mathbb{E}[\| \nabla f(\mathbf{x}) \|^2]\leq \epsilon$ is ${\cal O}\big( (n+\frac{1}{\sqrt{\epsilon}})\ln{\frac{1}{\epsilon}}\big)$, improving $\epsilon$ dependence over existing results. Numerical tests further validate the merits of proposed approaches.
Optimal supply chain and product design of biofuels
(2013-08) Marvin, William Alexander
Growth of a biomass-to-biofuels industry has the potential to reduce oil imports, support agriculture and forestry growth, foster a domestic biorefinery industry, and reduce greenhouse gas emissions compared to gasoline. Successful development of biofuels involves Process Systems Engineering challenges at various scales, including elucidation of complex chemical systems for upgrading biomass in terms of mechanisms, kinetics and thermochemistry, design of novel reactors and reactor networks, synthesis and optimization of novel process flow sheets, and supply chain optimization at the enterprise level. None of these aspects exist in isolation; each choice impacts the others and has an important role in the overall economic potential. The aim of this thesis has been to approach these multi-scale challenges by developing optimization models for biofuel supply chain and product design problems, specifically mixed integer linear programs. The biofuel supply chain optimization problems were formulated to determine economical and environmentally efficient biomass processing facility locations and capacities, simultaneously with biomass harvest and distribution. Focus was put on the production of biofuels in the Midwestern United States from grain, agricultural residues, energy crops and wood resources, and the feasibility of meeting governmental biofuel mandates in 2015. The product design problem investigated was for the production of blended gasoline with biomass-derived components. The strategy consists of i) constructing an exhaustive network of reactions consistent with an input set of chemistry rules and ii) using the network information to formulate and solve an optimization problem that yields an optimal product distribution and the sequence of reactions that synthesize them. This was applied to identify potential renewable oxygenates and hydrocarbons obtained from heterogeneous catalysis of biomass that can be blended with gasoline to satisfy ASTM specifications.
Performance, Throughput Properties, and Optimal Location Evaluation for Max-pressure Control
(2022-11) Barman, Simanta
Max pressure (MP) signal timing is an actuated decentralized signal control policy. Rigorous mathematical studies have proven stability or bounded total vehicle queues over a long period for all feasible demands. Those studies also established the theoretical benefits of different MP policies. However, the theoretical studies make some assumptions about traffic properties that may not represent reality, the effects of which are not explored much in the literature under realistic traffic conditions. The first portion of this study focuses on examining how different variations of MP control perform in realistic scenarios and finding the most practical policy among those for implementation in real roads. Microsimulation models of seven intersections from two corridors, County Road (CR) 30 and CR 109 from Hennepin County, Minnesota were created. Real life demand and current signal timing data provided by Hennepin County, Minnesota were used to make the simulations as close to reality as possible. Then, the performance comparisons of current actuated-coordinated (AC) signal control with an acyclic MP and two variations of cyclic MP policies are shown. The performance of different control policies in terms of delay, throughput, worst lane delay and number of phase changes are also presented. How different parameters affect performance of the MP policies is also presented. We found that better performance can be achieved with cyclic max pressure policy by allowing phase skipping when no vehicles are waiting. Findings from this study also suggest that most of the claimed performance benefits can still be achieved in real life traffic conditions even with the simplified assumptions made in the theoretical models. In most cases, MP control policies outperformed current signal control. The second portion of this study covers deployment strategies of MP control under limited budget and the associated stability properties. According to the theoretical results published so far, it can stabilize a network if all intersections are equipped with MP control for all stabilizable demands. However, budget constraints may not allow the installation of MP control on all intersections. Previous work did not consider a limited number of MP controlled intersections while proving the stability properties. Therefore, it is not clear whether a network can still be stabilized with a limited deployment of MP control. Using Lyapunov drift techniques, this thesis proves that even with a limited deployment, MP control can stabilize a network within feasible demand. Then, an optimization formulation to find the optimal intersections to install MP control given a limited budget is presented. We also present an efficient greedy algorithm to solve that optimization problem and prove that the algorithm solves the problem to optimality. Numerical results from simulations conducted on the downtown Austin network using an in-house custom simulator called AVDTA are then presented. The change in theoretical maximum servable demands for different amounts of deployments obtained from the optimization problem seemed to mostly match with simulation results. We found that limited deployment of MP control almost always performed better than random deployment of MP control in terms of servable stable demand. Average total queue length and link density were observed to decrease as the number of MP controls increased, which indicates better network performance. Average travel times per vehicle also decreased with installations of MP controls, which shows how the travelers would benefit from more MP controls.
Sparsity-promoting optimal control of power networks
(2016-12) Wu, Xiaofan
In this dissertation, we study the problems of structure design and optimal control of consensus and synchronization networks. Our objective is to design controller that utilize limited information exchange between subsystems in large-scale networks. To obtain controllers with low communication requirements, we seek solutions to regularized versions of the H2 optimal control problem. The proposed framework can be leveraged for control design in applications like wide-area control in bulk power systems, frequency regulation in power system/microgrids, synchronization of nonlinear oscillator networks, etc. The structure of the dissertation is organized as follows. In Part I, we focus on the optimal control problems in systems with symmetries and consensus/synchronization networks. They are characterized by structural constraints that arise either from the underlying group structure or the lack of the absolute measurements for a part of the state vector. Our framework solves the regularized versions of the H2 optimal control problems that allow the state-space representations that are used to quantify the system’s performance and sparsity of the controller to be expressed in different sets of coordinates. For systems with symmetric dynamic matrices, the problem of minimizing the H2 or Hinfinity performance of the closed-loop system can be cast as a convex optimization problem. Studying the symmetric component of a general system’s dynamic matrices provides bounds on the H2 and Hinfinity performance of the original system. Part II studies wide-area control of inter-area oscillations in power systems. Our input-output analysis examines power spectral density and variance amplification of stochastically forced systems and offers new insights relative to modal approaches. To improve upon the limitations of conventional wide-area control strategies, we also study the problem of signal selection and optimal design of sparse and block-sparse wide- area controllers. We show how different sparsity-promoting penalty functions can be used to achieve a desired balance between closed-loop performance and communication complexity. In particular, we demonstrate that the addition of certain long-range communication links and careful retuning of the local controllers represent an effective means for improving system performance. In Part III, we apply the sparsity-promoting optimal control framework to two problem encounters in distributed networks. First, we consider the optimal frequency regulation problem in power systems and propose a principled heuristic to identify the structure and gains of the distributed integral control layer. We define the proposed distributed PI-controller and formulate the resulting static output-feedback control problem. Second, we develop a structured optimal-control framework to design coupling gains for synchronization of weakly nonlinear oscillator circuits connected in resistive networks with arbitrary topologies. The structured optimal-control problem allows us to seek a decentralized control strategy that precludes communications between the weakly nonlinear Lienard-type oscillators.
Understanding Adaptivity in Machine Learning Optimization: Theories and Algorithms
(2022-05) Chen, Xiangyi
Optimization plays an indispensable role in modern machine learning, due to its necessity in many aspects, especially in model training. Over the past decade, the rapid development of research in deep machine learning models posed many new challenges for machine learning optimization. As a result, designing efficient and robust optimization algorithms remains an active research area within machine learning. In addition, some new and notable optimization algorithms were proposed to tackle the new challenges in model training. An important class of the new algorithms is motivated by the idea of incorporating adaptation into algorithm design, so that the algorithms can adapt to the local geometry of the optimization landscape. However, some of these newly designed algorithms with adaptation are tailored to achieve superior empirical performance for certain classes of optimization problems but are not well understood theoretically. Thus, the performance of these algorithms is less predictable in other domains or applications. In this thesis, we try to build theories for some algorithms with adaptation. In particular, the result of this thesis can be separated into three parts. In the first part, we analyze a class of algorithms with adaptation, which we call Adam-type algorithms, for nonconvex unconstrained optimization. We provide conditions for these algorithms to converge and shed light on design principles for this class of algorithms. In the second part, we extend the previous analysis to zeroth-order constrained/unconstrained optimization and propose an algorithm called ZO-AdaMM, which has superior performance in generating black-box adversarial attacks. In the third part, we study the gradient clipping operation for differentially private SGD. Gradient clipping adds a form of adaptation to SGD that can potentially hurt convergence. We identify regimes where gradient clipping is not an issue and verify the existence of these regimes in practice. Further, we provide a perturbation mechanism to mitigate the adversarial effect caused by gradient clipping.
Versatile Geometry Optimization with Hard Constraints
(2022-02) Overby, Matthew
Constrained numerical optimization is ubiquitous in geometry processing and simulation. For example, it can be used to minimize distortion between shape mappings or finding equilibrium in a dynamical system. Many important real-world problems are nonlinear and require sophisticated methods to optimize. However, there are several challenges associated with common applications. Models used for natural materials and distortion metrics are prone to numerical instabilities. Large domains give rise to computationally expensive operations, such as repeated factorizations of linear systems. For volumetric meshes, generating a feasible starting point of the optimization can be non-trivial. Constraints that model global behavior such as avoiding interpenetration are difficult to resolve robustly and efficiently. For an accurate representation, hard constraints are necessary to enforce certain behavior during the optimization. These difficulties have been the focus of geometry optimization literature for many years. Proximal methods provide a robust and scalable approach to nonlinear optimization, in which constraints and energies are represented as proximal operators. The alternating direction method of multipliers (ADMM) has especially grown in popularity for computer graphics applications. This work shows how ADMM can be modified to resolve many of the challenges associated with geometry optimization. We introduce a novel algorithm that applies ADMM to the robust simulation of hyperelastic deformation and mesh parameterization. However, ADMM alone does not produce a feasible solution since its reliance on proximal operations means it may never fully satisfy constraints. Toward addressing this limitation we provide formulations for resolving hard, nonlinear inequality constraints while making use of efficient precomputation. The effectiveness of these methods are demonstrated with several complex examples, including implicit time integration with collision resolution, volumetric shape mapping, and globally injective mesh deformation.

University Digital Conservancy

Browse by Subject

Browsing by Subject "optimization"