5 ms·
Related question: what's the minimum GPU that's roughly equivalent to Microsoft's Copilot+ spec NPU? I imagine that Copilot+ will become the target minimum sp
by bryanlarsen 2y ago
Related question: what's the minimum GPU that's roughly equivalent to Microsoft's Copilot+ spec NPU?
I imagine that Copilot+ will become the target minimum spec for many local LLM products and that most local LLM vendors will use GPU instead of NPU if a good GPU is available.
- kevinkeller 2y agoThe NPU in the Snapdragon SoC used by the Windows Surface laptops was quoted to be ~ 40 trillion ops/s (TOPS). Nvidia 4070 Ti has roughly the same performance: https://www.techpowerup.com/gpu-specs/geforce-rtx-4070-ti.c3950 https://www.techpowerup.com/gpu-specs/geforce-rtx-4070-ti.c3... Of course, I'm massively oversimplifying, but it should be in the ballpark.
- artemisart 2y agoNo, the Nvidia 4070 Ti has much higher performance, TOPS is for integer operations, the 4070 Ti has ~40 float32 TFLOPS and 641 TOPS https://www.nvidia.com/fr-fr/geforce/graphics-cards/40-series/rtx-4070-family/ https://www.nvidia.com/fr-fr/geforce/graphics-cards/40-serie... (which I would say would be peak TOPS for int4 operations, comparing it to the 4080 datasheet, and a bit more than half that for int8 operations) https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvid... page 34. I did not find the datasheet for 4070 Ti.
- tda 2y agoI was looking out for a new laptop but was wondering the same. This NPU thing might be one of Microsoft's bets that pays off, and makes all pre-NPU hardware obsolete quickly. Though of course they have doubled down on various failed projexts before (arm Windows, windows phones, etc)
- imtringued 2y agoBasically any GPU with at least 32GB RAM and 12 TFLOPs.