3 ms·
An AI chip on laptop devices would be amazing!
by alexander2002 2y ago
An AI chip on laptop devices would be amazing!
- ta988 2y agoThey already exist. Nvidia GPUs on laptops, M series CPUs from Apple, NPUs...
- alexander2002 2y agooh damn guess i am so uninformed
- viraptor 2y agoIt's pretty much happening already. Apple devices have MPS. Both new Intels and Snapdragon X have some form of NPU.
- moffkalast 2y agoIt would be great if any NPU that currently exists was any good at LLM acceleration, but they all have really bad memory bottlenecks.
- aurareturn 2y agoFirst NPU arrived 7 years ago in an iPhone SoC. GPUs are also “AI” chips. Local LLM community has been using Apple Silicon Mac GPUs to do inference. I’m sure Apple Intelligence uses the NPU and maybe the GPU sometimes.