SmaQ: Smart Quantization for DNN Training by Exploiting Value Clustering
Published in IEEE Computer Architecture Letters, 2021
Citation: Nima Shoghi*, Andrei Bersatti*, Moinuddin Qureshi, Hyesoon Kim, IEEE Computer Architecture Letters, 2021. https://doi.org/10.1109/LCA.2021.3108505
Introduces Smart Quantization (SmaQ), a quantization scheme that leverages the normal distribution properties of neural network data structures, leading to a memory usage reduction of up to 6.7x during training, with minimal impact on accuracy.