Hardware software co-design of machine learning accelerators using univariate functions

Abstract

As neural networks grow in complexity, efficient hardware-software co-design is crucial for balancing performance and resource constraints, especially for edge devices and FPGA accelerators. This thesis investigates optimization techniques for hardware-awaretraining and model compression using univariate functions. First, we optimize hardware using a simple constant coefficient multiplier on Hybrid Binary-Unary (HBUNN) architecture, which offers variable hardware costs for constant coefficients. By applying regularization-based training, we reduce the full model hard- ware area cost by 59.3% while improving accuracy by 0.42% over L2 regularization. Using this optimized model, the fully parallel-pipeline HBUNN-based ResNet-18 has a reduced area cost by 29.6% compared to conventional binary architectures. Next, we introduce Personal Self-Attention (PSA), a novel method for learning non- linear univariate functions. Using PSA with linear transformations, we demonstrate a 2×compression in hidden size of Multi-Layer Perceptrons (MLP), while matching accuracy. Applying this to an MLP-based vision model on CIFAR-10, we cut the number of operations by 45%–28%, boosting hardware efficiency. We validate PSA with two hardware accelerators while maintaining the same reference accuracy. First, an unrolled streaming design that reduces LUT + DSP usage by 25% while doubling throughput to 32kFPS. Second, a fixed-size SIMD accelerator that improves throughput by 62.1% while using only 3.5% additional LUTs.