4 ms·
Aren't they used for speech recognition -- for dictation? Also for FaceID. They're useful for more things than just LLM's.
by crazygringo 2y ago
Aren't they used for speech recognition -- for dictation? Also for FaceID.
They're useful for more things than just LLM's.
- JohnFen 2y agoYes, but I'm not interested in those sorts of uses. I'm wondering what else an NPU could be used for. I don't know what an NPU actually is at a technical level, so I'm ignorant of the possibilities.
- ItsBob 2y agoI'm probably about to show my ignorance here (I'm not neck-deep in the AI space but I am a software architect...) but are they not just dedicated matrix multiplication engines (plus some other AI stuff)? So instead of asking the CPU to do the math, you have a dedicated area that does it instead... well, that's my understanding of it. As to why, I think it's along the lines of this: the CPU does 100 things, one of those is AI acceleration. Let's take the AI acceleration and give it its own space instead so we can keep the power down a bit, add some specialization, and leave the CPU to do other stuff. Again, I'm coming at this from a high-level as if explaining it to my ageing parents.
- JohnFen 2y agoYes, that's my understanding as well. What I meant is that I don't know the fine details. My ignorance is purely because I don't actually have a machine that has an NPU, so I haven't bothered to study up on them.