3 ms·
An "NPU" is a matrix multiplier accelerator. It removes some general-purpose stuff that GPUs provide in favor of more "AI"-useful units, like support for values
by colejohnson66 1y ago
An "NPU" is a matrix multiplier accelerator. It removes some general-purpose stuff that GPUs provide in favor of more "AI"-useful units, like support for values a byte or smaller (i.e., FP4, INT4, etc.).
- cubefox 1y agoAll three of them accelerate matrix multiplications actually.
- almostgotcaught 1y agoany thing that computes matmul faster than by hand technically accelerates matmul - so what's your point?
- MobiusHorizons 1y agoI think NPUs are often aimed at efficient matmul performance. Not all implementations are significantly faster than vector units in the CPU, but they use much lower power. Gpu acceleration is typically much faster than the CPU, but also higher power.