3 ms·
Apple and Nvidia have arguably already diverged at the important forks in the road. Nvidia believes in complex GPU/streaming multiprocessor stacks - Apple relie
by nobankai 2y ago
Apple and Nvidia have arguably already diverged at the important forks in the road. Nvidia believes in complex GPU/streaming multiprocessor stacks - Apple relies on specialized hardware. Apple used to support a more complex software stack that could enable them to compete with Nvidia, but abandoned it shortly before the crypto craze to focus on NPU hardware.
...and then NPUs sorta did nothing. They run a few tiny models, maybe, but for any "serious" inference tasks Apple will automatically prioritize your 10x more powerful GPU hardware. Oftentimes the GPU is more efficient too, depending on the task.
So now Apple has a choice to make. They can either attempt to scale-up the NPU hardware and leave it on-device as dark silicon 99% of the time, or they can renovate their GPU hardware to support complex GPGPU operations and axe the NPU altogether. Right now it seems like Nvidia has the right idea, Apple just needs to find out how to scale it down as well as they can.
- bentt 2y agoTheir private cloud for AI reads on the surface like the answer. If they can establish privacy then they can push the work to the edges. Again, I'm not saying the are. They're the only ones that could, though.
- nobankai 2y agoIf you just read my comment and arrived at the conclusion that private compute is the solution, you misunderstand the issue at hand. Networking compute isn't an issue. Compute architecture is.
- bentt 2y agoSure so then ask yourself what advantages could an NPU have over a general purpose GPU. Your assumption is that Apple has erred. However if security was top priority, then that could make NPU make sense in the long run. And then ask why security is so important… it becomes vastly more important when your device is operating on other people’s workloads.