It's better than the TOPS nonsense where accuracy is completely thrown out the window. The quantization schemes used by llama.cpp do not operate directly on int

It's better than the TOPS nonsense where accuracy is completely thrown out the window. The quantization schemes used by llama.cpp do not operate directly on integers. They dequantize back to floats at runtime. You have to forgive me, but these NPUs were specifically designed for inference. You can indeed use them as general purpose accelerators, but I think you will struggle to find an application for them.