PermNet: Permuted Convolutional Neural Network

Abstract

Convolution filters in CNNs extract patterns from input by aggregating information across height, width and channel dimensions. Information aggregation across height and width dimensions performed using depthwise convolution, helps identify neighborhood patterns and hence is very intuitive. However the method in which channel dimension information is aggregated by channel summation seems mathematically simplistic and out of convenience. In this project we attempt to improve the channel dimension aggregation operations. The first approach introduces weighted summation channel aggregation in convolutions. The second approach introduces permuted convolutions which attempt to perform psuedo-width scaling by generating new constrained filters from existing filters. Implementing permuted convolutions comes with many challenges such as permutation explosion, stochasticity, higher memory and computation requirements. To resolve these issues, we come up with multiple variants of permuted convolutions and present their advantages and disadvantages. Lastly, we provide empirical results showcasing the performance of weighted channel summation networks and permuted convolution networks, present our findings and recommendations for future work.