4 ms·
It's an area of the chip suited for operations on low precision/range floating point numbers. Neural networks don't generally require high precision in floating
by jlouis 5y ago
It's an area of the chip suited for operations on low precision/range floating point numbers. Neural networks don't generally require high precision in floating point computations, but require a lot of them. This means your data paths can be smaller (16 bit wide rather than 32 bit wide); the consequence of which is you can do far more computation per mm2 die space.
The second part of the chain is that you also tailor the operations the die area support to those operations necessary in a typical neural network, further optimizing the chip.
The end result is very power-efficient execution of neural networks, which allows you to beat a GPU or CPUs power curve, improving thermals of the core, and in the case of mobile devices, optimizes battery usage.
- acdha 5y agoIt's probably also worth noting that the last part is fairly important to Apple since they have based a lot of their privacy stance around on-device processing of things like Siri commands, photo/video analysis, image recognition (VoiceOver can attempt to automatically describe images for blind people), speech to text dictation, the various audio/video enhancements like Center Stage or the voice emphasis features, etc. and all of that means they're running a lot of networks on battery. Those efficiency wins are almost certainly worth it even if third-party developers don't use it much.