[2A4-E-2] Machine learning: method extensions

[2A4-E-2-05] Final Sample Batch Normalization For Quantized Neural Networks

〇Joel Owen Nicholls¹, Atsunori Kanemura¹ (1. LeapMind Inc.)

Keywords:Neural network quantization, Batch normalization, Convolutional neural network

We outline and conduct empirical study into the effectiveness of a modified version of batch normalization, for combination with quantized neural networks. The proposed method uses only the statistics of the final batch for determining the batch normalization operation at the inference stage. This contrasts with the usual implementation, where population statistics are accumulated over many batches of training. The proposed and existing methods are compared over several models and datasets, which span both classification and object detection tasks. Overall, the proposed method exceeds the value and consistency of test performance compared to the usual batch normalization, in the case of quantized networks. For float precision networks, the usual method is best.