On the Effectiveness of Neural Networks Classifying the MNIST Dataset

Abstract

Convolutional Neural Networks (CNNs) are the primary driver of the explosion of computer vision. Initially popularized by AlexNet's performance in the ImageNet Competition in 2012, convolutional neural networks have since far-surpassed the traditional `hand-wired' models that were previously used in computer vision. They have been a focus of major investment and research from major institutes such as Google and OpenAI. This project is part 1 of a 2 part project researching potential optimizations of CNNs in the areas of convergence, processing speed, over fitting and accuracy. The first semester of the project implemented several optimizations from literature and combined them with CNNs to analyze their effectiveness. It also lays the groundwork for the second semester of research, which will be focused on combining recurrency from Recurrent Neural Networks (particularly Long Short-Term Memory (LSTM) networks.