This is such a confused blogpost I swear this had to be a teenager just banging their head against the wall. Wanting to natively train a quantized neural netwo

This is such a confused blogpost I swear this had to be a teenager just banging their head against the wall. Wanting to natively train a quantized neural network is stupid unless you are training directly on your microcontroller. I was constantly waiting for the author to explain their special circumstances and it turns out they don't have any. They just have a standard TinyML [0] use case that's been done to death with fixed point quanitization aware training, which unlike what the author of the blog post said, doesn't rely on terabytes of data. QAT is done on a conventionally trained model with much less data than the full training process. Doing QAT early has no benefits. The big downside of QAT isn't that you need a lot of data, it's that you need the same data distribution as the original training data and nobody has access to that, because only the weights are published. [0] https://medium.com/@thommaskevin/tinyml-quantization-aware-training-b4f29cdde787 https://medium.com/@thommaskevin/tinyml-quantization-aware-t...