4 ms·
I find this paper kind of "medium" (as the transformers baseline they compare against is glacially slow), but Intel's integration efforts in open source LLM run
by brucethemoose2 3y ago
I find this paper kind of "medium" (as the transformers baseline they compare against is glacially slow), but Intel's integration efforts in open source LLM runtimes are real and very interesting. See, for instance, their PRs in llama.cpp, not to speak of all their other open source efforts.
- baq 3y agoTheir software folks are really good, at least in the GPU space. The hardware seems stuck in past decade and the process woes don't help either - but the software should be ready if they ever dig themselves out of the hardware hole.
- selimthegrim 3y agoAt NeurIPS it seemed like the Intel Labs/materials research/hardware people were out in force
- brucethemoose2 3y agoI mean, if they sell me a 48GB GPU that's not $5K like a W7900, I will drop my Nvidia card faster than I can blink. I don't even care if its kinda slow and uses an older process. And then I may end up fixing Intel bugs in ML projects we use, and deploying them on Intel cloud hardware... which is hopefully more available as well.
- kiratp 3y ago+1 to this. I’ve been working with their research teams on this and credit due indeed. Some context: https://github.com/ggerganov/llama.cpp/issues/2555#issuecomment-1698302171 https://github.com/ggerganov/llama.cpp/issues/2555#issuecomm... https://github.com/ggerganov/llama.cpp/discussions/3965 https://github.com/ggerganov/llama.cpp/discussions/3965