6 ms·
Intel slaps forehead, says I got it: AI PCs. Sell them AI PCs
- debacle 3y agoThis doesn't seem like the worst idea, but from a hardware perspective I'm not sure how an "AI PC" differs from machines with nice GPUs.
- brookst 3y agoI’d welcome correction, but my understanding is that NPUs are “native” neural weight processors that trade the flexibility of doing graphics for the optimization of not having a layer like CUDA between neural net and hardware.
- mensetmanusman 3y agoThey are different enough that you could certainly add additional performance gains if you knew you were running LLaMa versus Fortnite.
- amelius 3y agoAn AI PC will have an AI App Store. Where Intel can reap like a 30% tax of all developer profits. Of course the hardware is the same, except for some features that allow Intel to lock it down hard.
- OJFord 3y agoYou could say the same about gaming PCs (ok, and LEDs). And less obviously about almost anything. So much is just marketing, including product design as a form of it.
- jdietrich 3y agoGPUs are increasingly optimised for machine learning workloads, but they aren't really designed for it on an architectural level and still leave a lot of potential performance and efficiency on the table. Current ML implementations are still quite kludgy and path-dependent, with lots of technical debt. There is huge potential for optimisation throughout the stack.
- unhammer 3y agohuh, Intel making GIMP plugins https://github.com/intel/openvino-ai-plugins-gimp https://github.com/intel/openvino-ai-plugins-gimp
- deleted 3y ago[deleted]
- sovietmudkipz 3y agoI'm not entirely sure that intel is in the best position to make consumer grade specialty "AI" PC because they make CPUs not GPUs. Maybe that's a bad interpretation on my end since intel has afforded graphics for computers without a dedicated GPU. Side note: as a hobbyist game dev I've learned about shaders. A shader is simply a program written for a GPU. In game dev we talk about shaders for graphics output but one can write generic shaders too (look up GPGPU). My eyes were opened to the concept that there are programs written for CPU and programs written for GPU. A CPU typically has <=12 cores and each core computes very quickly. A GPU typically has <=1024 cores and each core computes quickly (but not as quick as a CPU core). It follows that a program ideal for a GPU is one that benefits massive parallelization. I'm still noodling on what that ideal program could be, outside of the obvious graphics processing. Sharing in case others hadn't realized this tidbit and/or someone more experienced can share their thoughts.
- klohto 3y agoI got news for you brother, maybe one day will use it somewhere, matrix computations
- gumballindie 3y ago> I'm still noodling on what that ideal program could be, outside of the obvious graphics processing. Well machine learning does its computations on GPUs much like shaders. It's basically the same thing with a different function in the kernel: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.... "CUDA C++ extends C++ by allowing the programmer to define C++ functions, called kernels, that, when called, are executed N times in parallel by N different CUDA threads, as opposed to only once like regular C++ functions."
- derstander 3y ago> I'm not entirely sure that intel is in the best position to make consumer grade specialty "AI" PC because they make CPUs not GPUs. Intel makes (albeit recently) the Arc series of dedicated GPUs. I’m not a graphics programmer so it’s not quite clear how the logic units compare between nVidia, AMD, and Intel, but this link suggests thousands of shading units/cores. https://wccftech.com/intel-arc-a770-a750-a580-graphics-cards-official-specs-unveiled-up-to-32-xe-cores-16-gb-gddr6-2-1-ghz-clocks/amp/ https://wccftech.com/intel-arc-a770-a750-a580-graphics-cards...
- trey-jones 3y agoPurely marketing from front to back. It's a little bit similar to how AMD spent too much time on R&D for a "true quad-core" processor (which ended up being Phenom) back in the late aughts (2008 or so?). Intel kinda glued two dual-core CPUs together and beat them to market with a "quad-core" processor, securing dominance for about 15 years. I used the Phenom IIx4 for almost that entire 15 year period - it was really great! But they didn't sell that many. Unfortunately it seems like fiscal success in 2023 mostly means jumping on the hype train of the day and beating the drum for all it's worth, and being ready to pivot to the next thing quickly. Rather than maximizing actual value in your product.
- agentultra 3y agoWell crypto was starting to tank Nvidia's numbers, gotta keep growing! New thing: LLMs everywhere!
- firebat45 3y ago>Unfortunately it seems like fiscal success in 2023 mostly means jumping on the hype train of the day and beating the drum for all it's worth, and being ready to pivot to the next thing quickly. Rather than maximizing actual value in your product. We have nobody to blame but ourselves. Consumers are far too happy to just jump on the hypiest train and don't care enough to actually educate themselves about which product has maximum value. It's the entire reason Apple still exists as a company, for example.
- digging 3y ago> We have nobody to blame but ourselves. Consumers are far too happy to just jump on the hypiest train and don't care enough to actually educate themselves about which product has maximum value. It's the entire reason Apple still exists as a company, for example. And water is all too happy to flow downhill. So we build dams to hold it back.
- deleted 3y ago[deleted]
- user3939382 3y ago
- itsoktocry 3y agoThe idea isn't terrible, but I don't understand the market. People do 90% of their work "on the web" or "in the cloud", but they're going to be concerned about running an LLM locally (and have no ability to audit security)?
- daveguy 3y agoAre you implying that running an LLM locally means having no ability to audit security vs running one "in the cloud"? Seems like the opposite to me.
- belltaco 3y agoI think they meant people trust the web enough to use it for most of their activities so the ability to audit security in a local LLM may not be a big selling point.
- UncleMeat 3y agoNo the point is that I'm already doing basically all of my computing (both work and non-work) via web applications. All of the AI integrations into products can happily run on backends rather than locally on my machine. There are exceptions, of course, but this is the state of the bulk of modern computing.
- itsoktocry 3y ago>having no ability to audit security vs running one "in the cloud"? Seems like the opposite to me. By "ability" I mean, literally, ability. If the customer is someone who does most of their computing in the cloud today, what value is a "personal secure LLM"? Can this person even tell if their local LLM is secure? Doubt it. For the average person, that is best left to "the cloud".
- wongarsu 3y agoIndividuals aren't concerned about that, but plenty of companies and government agencies are. Most wouldn't even want to audit the LLM, they just want to audit that the information doesn't leave their network. Running the LLM locally on some "AI PC" would satisfy that perfectly, and the types of organizations that care the most about this are also the ones with the most generous budgets for these kinds of problems.
- adolph 3y agoApple's been making hay with "neural engines" in their hardware for quite a while. It is curious why Intel hasn't made a bigger splash with its efforts, which go back a ways and have included (semi?) cancelled things like the RealSense from aquisitions like Movidius (whose chips also formed the core of early products like the Google AIY Vision Kit; AIY still being marketed as a deployment target for Coral). I guess that: 1. Apple has been able to capitalize on their constrained stack of HW/SW. 2. Intel has struggled with being the unpopular dish at a well optioned buffet. On-device ML inferencing is going to be an interesting space. Your iPhone, iPad, Mac, and Apple TV make use of a specialized neural processing unit called Apple Neural Engine (ANE) that's way faster and more energy efficient than the CPU or GPU. https://www.makeuseof.com/what-is-a-neural-engine-how-does-it-work/ https://www.makeuseof.com/what-is-a-neural-engine-how-does-i... This year at WWDC 2022, Apple is making available an open-source reference PyTorch implementation of the Transformer architecture . . . specifically optimized for the Apple Neural Engine (ANE), the energy-efficient and high-throughput engine for ML inference on Apple silicon. https://machinelearning.apple.com/research/neural-engine-transformers https://machinelearning.apple.com/research/neural-engine-tra... Intel RealSense Technology, formerly known as Intel Perceptual Computing, was a product range of depth and tracking technologies designed to give machines and devices depth perception capabilities. . . In August 2021 Intel announced it was "winding down" its RealSense computer vision division to focus on its core businesses. https://en.wikipedia.org/wiki/Intel_RealSense https://en.wikipedia.org/wiki/Intel_RealSense The AIY Maker Kit . . . Using a Raspberry Pi and a few accessories, our tutorials show you how to build a portable device that can identify objects, people, and body poses with a camera, and recognize voice commands or other sounds with a microphone. . . [using] TensorFlow Lite models. . . . https://aiyprojects.withgoogle.com/maker/ https://aiyprojects.withgoogle.com/maker/
- lumost 3y agoI think folks are underestimating how far we have to go on inference performance. Even ignoring LLM’s like phi-1.5. PyTorch was meant to be easy to use first, and fast second. This turned out to be the winning combo for adhoc science workloads. For pure LLM inference, you can go much further by fusing operations together and eventually building asic extensions to improve transformer perf. I’d easily believe that we will see a 10x reduction in inference costs over the next 5 years as hardware folks catch up.
- brucethemoose2 3y ago> PyTorch was meant to be easy to use first, and fast second. People have been saying this for years, yet most training and a majority(?) of inference is done in PyTorch. We have all sorts of cool ML compilers, some that directly support importing PyTorch projects, largely sitting there gathering dust. Even torch.compile is catching on like molasses in winter.
- TeMPOraL 3y ago> People have been saying this for years, yet most training and a majority(?) of inference is done in PyTorch. The usual symptom of "cheaper to waste more resources than do it properly"?
- brucethemoose2 3y agoThat, but I think many users don't even know its an option to try. I am hoping torch 2.1 improves this situation. torch.compile should "just work" much more often with `dynamic=true`, and its widespread usage might make people wonder "what other cost-reducing compilation frameworks are out there?"
- eep_social 3y agoI think the changed cost of money will improve this over time. It’s possible interest rates will swing back (and there is still a lot of denial about how far back) but IMO the days of throwing money at things and punting on efficiency appear to be waning.
- sys32768 3y agoA PC with a built it AI "IT guy" would be a game changer. It sits there crunching all the logs and performance data, making tweaks or recommendations for solving known or potential issues. Much of what I did in my systems support was sifting through log files trying to find the answers amid mountains of crappy search results.
- 0xbadcafebee 3y agoOne part of me can't wait for this hype cycle to be over. But another part of me wants to make bank off all these idiots investing in AI.
- davidy123 3y agoI've learned the Register isn't a very good rag, they push the cattiness so much, you can only get information from between the lines. The reality is "AI" is an important part of the future, local AI if it's going to be a good future. The other reality is Apple is way ahead with its M chips for consumer devices, with their tremendous memory bandwidth and GPU well suited to running LLMs up to the speed of a 4080. If they didn't limit RAM so much and could scale up to the level of professional GPUs, they would be very compelling. AMD is being very flaky with their rollout. They only provide the ML chip on some of their new chips, and let some vendors block it out in BIOS. I've been watching their github demo site, and they won't commit to anything but a very limited demo for Windows. Intel at least released an open source kit earlier this year and will support their VPU on all Meteor Lake+ chips, as well as other processors. Consistency and openness is needed to break NVidia's deadlock here, OpenVINO could be a good step. Meteor Lake will have 128GB/s for on-chip RAM bandwidth, that's a fraction of M-chips, it really remains to be seen how this will shake out, but it seems likely there will be a new class of computing device, and it may make all others very second class.
- Dylan16807 3y agoThe cattiness is fine, because the validity of the use case doesn't mean their chips have a big enough advantage on that use case to matter. Not just because of all the vector units you already have in the CPU and iGPU cores, but as another comment points out memory bandwidth is often the bottleneck and a bunch of matrix multiply boosts won't affect that.
- davidy123 3y agoYes it is useful as a deflection against early hype, but I don't think the headline is fair.
- darklycan51 3y agoHe's not wrong though, my next pc is gonna be purchased with running LLMs locally, just in case.
- UncleOxidant 3y agoStill, that's a very niche market, don't you think?
- marshray 3y agoLast I heard, ChatGPT is the most rapidly-adopted app in the world. If not this, then what?
- UncleOxidant 3y agosure, but not a lot of people are going to be running it on their own PC at home - that's what I meant by niche market for hardware.
- sproketboy 3y ago[dead]
- bitwize 3y agoUntil I can get my own Conrad (from the Choose Your Own Adventure book Supercomputer[0]), I'm gonna press X to doubt on the value add of this. [0] https://www.goodreads.com/book/show/755062.Supercomputer https://www.goodreads.com/book/show/755062.Supercomputer