Deterministic Binary Filters for Convolutional Neural Networks

Deterministic Binary Filters for Convolutional Neural Networks

Vincent W.-S. Tseng, Sourav Bhattacharya, Javier Fernández Marqués, Milad Alizadeh, Catherine Tong, Nicholas D. Lane

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 2739-2747. https://doi.org/10.24963/ijcai.2018/380

We propose Deterministic Binary Filters, an approach to Convolutional Neural Networks that learns weighting coefficients of predefined orthogonal binary basis instead of the conventional approach of learning directly the convolutional filters. This approach results in model architectures with significantly fewer parameters (4x to 16x) and smaller model sizes (32x due to the use of binary rather than floating point precision). We show our deterministic filter design can be integrated into well-known network architectures (such as ResNet and SqueezeNet) with as little as 2% loss of accuracy (under datasets like CIFAR-10). Under ImageNet, they result in 3x model size reduction compared to sub-megabyte binary networks while reaching comparable accuracy levels.
Keywords:
Machine Learning: Deep Learning
Computer Vision: Computer Vision