4 ms·

I wouldn't call FP16 "FLOPS". Is anyone using them for non-AI workloads (e.g. compute)?

• imtringued 2y ago
  It's better than the TOPS nonsense where accuracy is completely thrown out the window. The quantization schemes used by llama.cpp do not operate directly on integers. They dequantize back to floats at runtime. You have to forgive me, but these NPUs were specifically designed for inference. You can indeed use them as general purpose accelerators, but I think you will struggle to find an application for them.
• touisteur 2y ago
  Yeah I wish we'd start to say what kind of FLOPS one's talking about. And don't start me on 'AI FLOPS' or TOPS...