SmaQ: Smart Quantization for DNN Training by Exploiting Value Clustering

Published in IEEE Computer Architecture Letters, 2021

Citation: Nima Shoghi*, Andrei Bersatti*, Moinuddin Qureshi, Hyesoon Kim, IEEE Computer Architecture Letters, 2021.

Introduces Smart Quantization (SmaQ), a quantization scheme that leverages the normal distribution properties of neural network data structures, leading to a memory usage reduction of up to 6.7x during training, with minimal impact on accuracy.

Access paper here