low bit depth quantized puts GPU's at quite a disadvantage, since they can't use their massively parallel floating point units, and often spend a lot of time pa

low bit depth quantized puts GPU's at quite a disadvantage, since they can't use their massively parallel floating point units, and often spend a lot of time packing and unpacking numbers. However, minor tweaks to the design of GPU's could probably make them 5x quicker at directly processing quantized data (a 4 bit multiplier is so small you can afford to do thousands of them per clock cycle in a tiny silicon area), and I'm sure the next generation of GPU's will have that.