5 ms·
Intel Meteor Lake's NPU
- ein0p 3y agoLooks like they repurposed Movidius SHAVE architecture and added a systolic MXU. Makes sense, if they can sort out their software story
- marssaxman 3y agoThat is the big "if", to be sure. I spent three years working for Intel after they acquired Vertex.AI, trying to help build that software story, and it was rough going; we didn't accomplish much, and it doesn't sound like they're using any of it.
- ein0p 3y agoI was impressed by their _CPU_ kernels in OpenVINO which I studied very closely after discovering I cannot beat their performance even on kernels specialized for my specific tensor shapes. Turns out the reason I can't beat them is because they jitted their kernels with xbyak, and specialized them for those shapes, too. They also obviously knew exactly how to maximize the ALU utilization, and counted every cycle, something that's very hard to do unless you specialize _just_ on CPU performance and have access to the internal documentation. Top notch work. But that team used to be in Russia back then, in Nizhny Novgorod, IDK if it's still a part of Intel.
- vient 3y agoLooking at OpenVINO repo activity, seems that most of them got relocated and still work at Intel.
- touisteur 3y agoYes the _CPU_ perf of OpenVINO is a thing of beauty. Debuggability is a nightmare story but if it works you're GOLD. The GPU/Myriad story was very clunky last times I tried.
- ein0p 3y agoI hope some Intel PHB reads this and purely stochastically makes the right decision for the first time in his/her life. It seems pretty existential to be competitive at selling shovels during a gold rush.
- touisteur 3y agoThere's a Matt Pharr-level story to tell here if you feel like it one day. I'm also looking at what the very cool maxas authors became. Intel seems like a dark hole of blackness and holeness for acquired tech these days.
- fpgamlirfanboy 3y ago1024 fp16 macs is pretty good but the 128b vector datapath is weak sauce. on the other hand 2MB SRAM is legit. i wonder how many tiles (i don't think it's in the post).
- ein0p 3y agoI think the main advantage of Movidius was its memory architecture. They had to disclose the HW arch for some sort of a tender in the EU which is how I stumbled upon their HW documentation years ago. Basically IIRC even though you’re right and 128 bit is weak sauce, the strength of that arch was that the memory was much “closer” to the cores and it was partitioned and accessed by multiple cores at the same time, boosting overall available bandwidth. The weakness of it was that it was (IMO) overly complicated for no good reason, and imposed a rather inflexible programmability model which if their software layer didn’t do certain things, you couldn’t do them at all. Which was a problem before transformers, because models tended to use a greater variety of ops
- AnotherGoodName 3y agoThe article talks about ai development workloads and how there's little need for a low power low performance accelerator compared to a high end GPU but I think the point is end user workloads. Software is coming with AI integrations. IDEs, spreadsheets, word processors, graphics programs. These absolutely will benefit from low power ai acceleration. Sure if i want best in class acceleration I'll buy the damn 300w GPU. But really I just want some autocompletion without burning my lap or requiring cloud connectivity.
- Dalewyn 3y agoI always feel that accelerators are in an awkward spot, because at some point with a sufficiently powerful CPU most workloads finish quickly enough. And let's be real, even lowly Core i3s these days are overkill for most people; let alone mid tier offerings like Core i5s which are the most popular for commercial users.
- 7speter 3y agoIsn’t a big advantage of an accelerator is that its more optimized to do this sort of work than a cpu? This npu is a tiny dot on the Meteor Lake die while the cpus take up a whole tile and accounts for much of the power usage of the soc at any given time.
- wtallis 3y agoI don't think the NPU is tiny, just much more power efficient than the GPU or CPU for the right kind of tasks.
- acdha 3y agoThat’s the idea but it looks like this might not pull it off unless Intel aggressively gives away optimized libraries. If it’s too hard to get to the point where it’s actually competitive, most developers are going to use the faster iGPU.
- touisteur 3y ago
- brcmthrowaway 3y agoLooks like a great competitor to apple neural engine!
- int0x29 3y ago"Instead of building a custom command processor, Intel uses Movidius’s 32-bit LEON microcontrollers. LEON use the SPARC instruction set and run a real time operating system." Your Intel core now runs two ISAs and three OSs. Managment engine runs Minix (and used to use the ARC isa).
- deleted 3y ago[deleted]
- fpgamlirfanboy 3y agothis is very common. lots of companies around (e.g. cadence) that sell SIPs as synthesizable RTL for easy integration into chip designs (and lots of hardware companies buy these SIPs).
- stefan_ 3y agoIndeed, silicon design is long in its NPM left-pad phase. Which begs the question why it is still so open-source averse and patent-pilled.
- userbinator 3y agoIt's always weird to see Intel using "foreign" ISAs, but not too surprising for something they acquired. The ME's use of SPARC and ARC before that is more unusual; they could've at least used a 386/486-class core, or even the i860/960 if they wanted something RISC.
- int0x29 3y agoPretty sure ME is Intel Quark based now. Quark is intel's weird 32 bit x86 as a microcontroller chip.
- rbanffy 3y agoA 32-bit Linux-capable microcontroller at that. I still keep my Galileo board. Will probably assemble it into a picture frame and run a heterogeneous cluster with it and some of the other boards I accumulated over the years. This is Gibson, the 4 RPi Zero W's running Docker Swarm: https://twitter.com/0xDEADBEEFCAFE/status/1729863007755444620 https://twitter.com/0xDEADBEEFCAFE/status/172986300775544462...
- noelwelsh 3y agoSo how do you program these things? I tried looking into the AMD equivalent and the development story was very primitive. Is the future going to be programming against a different opaque binary blob for each NPU one wants to support?
- lunfardl 3y agoit is prob meant to be used through OpenVINO which is open-source, so I don't expect an obscure blob.
- tukantje 3y agoI just bought a new computer for the love of lord...
- automatic6131 3y agoIt's okay, his takeaway at the bottom is any PC with a midrange GPU is already an "AI PC".
- ZuLuuuuuu 3y agoI am sure Intel will improve the shortcomings in the next generation, so it is a good start. But I wonder how this will compare with Qualcomm's new PC CPU (Snapdragon X Elite) which also has its own NPU.
- downrightmike 3y agoThey should all be equivalent by the end of the year, Qualcomm is launching this fall, Intel has plans, but we know about how well they deliver.
- majke 3y ago> Even economy class plane seats have power outlets these days. Hopefully Intel will iterate on both hardware and software to expand NPU use cases going forward. I can totally see myself doing inference on my phone, but arguably - this is not intel's territory.
- vient 3y agoNot the best time for testing OpenVINO at least, they plan to release NPU support in 2024.1 as I understand, current version 2024.0 has very limited NPU support if any.
- wiseowise 3y ago> Even economy class plane seats have power outlets these days. “Do you guys not have power outlets? Yeah, everybody’s got a power outlet.” What a time to be alive.