3 ms·
Its desperately needed though. If AMD or Intel started going around github and writing in acceleration for random popular ML repos (and not just one off abando
by brucethemoose2 3y ago
Its desperately needed though.
If AMD or Intel started going around github and writing in acceleration for random popular ML repos (and not just one off abandoned demos), that would change so much.
- refulgentis 3y agoThese models run on CUDA. Making a version that ran on Metal filled a feature gap on Apple's proprietary GPUs. CPUs aren't involved, the last thing, for example, Stable Diffusion needs is help from AMD and Intel.
- zeusk 3y agoCPUs are actually involved, especially with SIMD like AVX and VNNI instructions. It all depends on the scale of your compute. https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/overview.html https://www.intel.com/content/www/us/en/developer/tools/open...
- refulgentis 3y agoLinking to an Intel SDK with “deep learning” in it does prove it’s possible to do matmuls on CPUs. But this a nitpicked to death thread: it’s good to see Apple releasing research, the Metal stuff was to make it run on their GPU, and there simply aren’t any relevant uses of CPUs. Academically, yes, CPUs have SIMD so they can implement the same algorithm that drives GPU perf, but without some massive breakthrough the one that can do more matmuls wins, and nVidia wins massively, in theory and practice.
- zeusk 3y agoI'm not just linking it as a proof, I now work at Intel and collaborate with Microsoft on bringing new platform technologies to life. I'm currently involved in VNNI and hybrid core scheduling KPIs. OpenVINO allows developers to target execution engines on client based on their needs and CPU is a well supported engine for lightweight inferencing that is latency sensitive. Matmuls (FMAC especially) did evolve as a SIMD workload on CPUs but it's still forcing vector execution on cores designed around scalar workloads - you just have to be mindful of tradeoffs between executing a bunch of AVX/VNNI instructions for inferencing vs setting up a GPU/VPU/IPU context and performing that inference asynchronously. AFAIK, ARM is also interested in use of NEON/SIMD co-processors as low latency inference engines. Microsoft and Intel have a lot of interesting stuff in the works for AI compute (as I'm sure so do Apple, Nvidia and to a lesser extend AMD).
- sebzim4500 3y agoYou know that Intel makes GPUs right? In fact, they are by far the biggest producer of GPUs by volume. Sure, they aren't being used in datacenters but if you are making an application that needs to run locally then you absolutely need good support for intel GPUs.
- cinntaile 3y agoYou're talking about integrated GPUs I presume?
- sebzim4500 3y agoMainly yes, although they do also make dedicated gpus
- dragonwriter 3y agoiGPUs aren’t necessarily the worst solution where the problem is one where “can I do it at all” is often constrained by RAM available to the GPU, even if dGPUs have better compute performance.
- astrange 3y agoApple GPUs are also integrated. (But "unified memory" is a better technical distinction.)
- girvo 3y agoIntel actually just released a set of data centre GPUs too
- brucethemoose2 3y agoAMD and Intel both ship a lot of GPUs. In fact there are good AMD/Intel implementations, just not enough dev effort to add them to popular SD projects
- dragonwriter 3y agoIn Intel’s case, they have their own SD-based project (OpenVINO AI Plugins for GIMP), instead of putting effort into popular SD projects.