5 ms·

The article talks about ai development workloads and how there's little need for a low power low performance accelerator compared to a high end GPU but I think

The article talks about ai development workloads and how there's little need for a low power low performance accelerator compared to a high end GPU but I think the point is end user workloads. Software is coming with AI integrations. IDEs, spreadsheets, word processors, graphics programs. These absolutely will benefit from low power ai acceleration. Sure if i want best in class acceleration I'll buy the damn 300w GPU. But really I just want some autocompletion without burning my lap or requiring cloud connectivity.

Dalewyn 3y ago
I always feel that accelerators are in an awkward spot, because at some point with a sufficiently powerful CPU most workloads finish quickly enough. And let's be real, even lowly Core i3s these days are overkill for most people; let alone mid tier offerings like Core i5s which are the most popular for commercial users.
- 7speter 3y ago
  Isn’t a big advantage of an accelerator is that its more optimized to do this sort of work than a cpu? This npu is a tiny dot on the Meteor Lake die while the cpus take up a whole tile and accounts for much of the power usage of the soc at any given time.
  - wtallis 3y ago
    I don't think the NPU is tiny, just much more power efficient than the GPU or CPU for the right kind of tasks.
  - acdha 3y ago
    That’s the idea but it looks like this might not pull it off unless Intel aggressively gives away optimized libraries. If it’s too hard to get to the point where it’s actually competitive, most developers are going to use the faster iGPU.
    - touisteur 3y ago
      You hit the nail on the head. Software, libraries, dev tools, profiling resources, mindshare... And for Intel, a good deal of maintained interest. For those of us who used and programmed SHAVE cores, to see everything slowly close down behind openvino, no apparent interest from Intel in years, the writing was on the wall. Repeatedly, they taught us: use nothing from Intel except main-mainline x86_64.
      - acdha 3y ago
        What’s sad is that I remember the same lesson from Itanium: to preserve a small revenue stream from their compiler and optimized library business, they made almost every benchmark even worse at a time when they most needed businesses to be comfortable switching.
- parl_match 3y ago
  It makes systems "feel" way more responsive. It's a huge quality of life increase. Apple Silicon's user/product story has basically changed how people view this stuff forever.
  - The_Colonel 3y ago
    What does the Apple Silicon (NPU?) accelerator do? I mean, what is it used for?
    - QuantumSeed 3y ago
      - Face and object detection in photos and videos - Computational photography features like Night Mode and Smart HDR - Real-time video processing for effects and stabilization - Object and action recognition in videos - Faster on-device speech recognition and natural language understanding for Siri - Real-time translation, object detection, and scene understanding in AR
- userbinator 3y ago
  And let's be real, even lowly Core i3s these days are overkill for most people They would be, if it weren't for the egregious insanity of software bloat, particularly the web-based stuff.
- imtringued 3y ago
  4 TFLOPs is significantly faster than your CPU. Even 1.35 TFLOPS represents a reasonable edge. The question should be why Intel added another processor instead of extending AVX.
  - b112 3y ago
    Oh that's easy. The registers in this CPU are all quantumly entwined, with Intel having updates signed. Thus, Intel can one-way update one or all CPUs remotely. No Faraday cage will protect you. Further, this CPU has access to caches on all other cores, so Intel can dynamically inhibit, or modify code as they please. We'll find this out in 2027, I dare not say more, they'll track me through time and afg#.<NO CARRIER>
  - DeathArrow 3y ago
    >The question should be why Intel added another processor instead of extending AVX. It seems that efficient cores don't support much of AVX extensions. You have to turn those cores off if you want to use AVX512, for example.
    - vient 3y ago
      It worked only on first batches/microcode versions of hybrid CPUs, since then Intel had completely disabled AVX-512 on them.
  - Archit3ch 3y ago
    I wouldn't call FP16 "FLOPS". Is anyone using them for non-AI workloads (e.g. compute)?
    - imtringued 3y ago
      It's better than the TOPS nonsense where accuracy is completely thrown out the window. The quantization schemes used by llama.cpp do not operate directly on integers. They dequantize back to floats at runtime. You have to forgive me, but these NPUs were specifically designed for inference. You can indeed use them as general purpose accelerators, but I think you will struggle to find an application for them.

Dalewyn 3y ago

I always feel that accelerators are in an awkward spot, because at some point with a sufficiently powerful CPU most workloads finish quickly enough. And let's be real, even lowly Core i3s these days are overkill for most people; let alone mid tier offerings like Core i5s which are the most popular for commercial users.

7speter 3y ago
Isn’t a big advantage of an accelerator is that its more optimized to do this sort of work than a cpu? This npu is a tiny dot on the Meteor Lake die while the cpus take up a whole tile and accounts for much of the power usage of the soc at any given time.
- wtallis 3y ago
  I don't think the NPU is tiny, just much more power efficient than the GPU or CPU for the right kind of tasks.
- acdha 3y ago
  That’s the idea but it looks like this might not pull it off unless Intel aggressively gives away optimized libraries. If it’s too hard to get to the point where it’s actually competitive, most developers are going to use the faster iGPU.
  - touisteur 3y ago
    You hit the nail on the head. Software, libraries, dev tools, profiling resources, mindshare... And for Intel, a good deal of maintained interest. For those of us who used and programmed SHAVE cores, to see everything slowly close down behind openvino, no apparent interest from Intel in years, the writing was on the wall. Repeatedly, they taught us: use nothing from Intel except main-mainline x86_64.
    - acdha 3y ago
      What’s sad is that I remember the same lesson from Itanium: to preserve a small revenue stream from their compiler and optimized library business, they made almost every benchmark even worse at a time when they most needed businesses to be comfortable switching.
parl_match 3y ago
It makes systems "feel" way more responsive. It's a huge quality of life increase. Apple Silicon's user/product story has basically changed how people view this stuff forever.
- The_Colonel 3y ago
  What does the Apple Silicon (NPU?) accelerator do? I mean, what is it used for?
  - QuantumSeed 3y ago
    - Face and object detection in photos and videos - Computational photography features like Night Mode and Smart HDR - Real-time video processing for effects and stabilization - Object and action recognition in videos - Faster on-device speech recognition and natural language understanding for Siri - Real-time translation, object detection, and scene understanding in AR
userbinator 3y ago
And let's be real, even lowly Core i3s these days are overkill for most people They would be, if it weren't for the egregious insanity of software bloat, particularly the web-based stuff.
imtringued 3y ago
4 TFLOPs is significantly faster than your CPU. Even 1.35 TFLOPS represents a reasonable edge. The question should be why Intel added another processor instead of extending AVX.
- b112 3y ago
  Oh that's easy. The registers in this CPU are all quantumly entwined, with Intel having updates signed. Thus, Intel can one-way update one or all CPUs remotely. No Faraday cage will protect you. Further, this CPU has access to caches on all other cores, so Intel can dynamically inhibit, or modify code as they please. We'll find this out in 2027, I dare not say more, they'll track me through time and afg#.<NO CARRIER>
- DeathArrow 3y ago
  >The question should be why Intel added another processor instead of extending AVX. It seems that efficient cores don't support much of AVX extensions. You have to turn those cores off if you want to use AVX512, for example.
  - vient 3y ago
    It worked only on first batches/microcode versions of hybrid CPUs, since then Intel had completely disabled AVX-512 on them.
- Archit3ch 3y ago
  I wouldn't call FP16 "FLOPS". Is anyone using them for non-AI workloads (e.g. compute)?
  - imtringued 3y ago
    It's better than the TOPS nonsense where accuracy is completely thrown out the window. The quantization schemes used by llama.cpp do not operate directly on integers. They dequantize back to floats at runtime. You have to forgive me, but these NPUs were specifically designed for inference. You can indeed use them as general purpose accelerators, but I think you will struggle to find an application for them.