3 ms·

Linux on X Elite Laptops [video]

19 points by cl3misch 2y ago

marshray 2y ago
tl;dr: * Not supported out-of-the-box by distros. * Device support is not complete, some models are better than others. * He had to look at websites and mailing lists to get it working.
- daghamm 2y ago
  Well, these devices where just released, there is no official or even non-official support yet. But I think in 3 or 6 months things will look very different.
Havoc 2y ago
Linux is a good start but I’m really interested in when someone manages to accept the npu with an open stack. ie will a smallish model say 9B or so run at reasonable pace
- marshray 2y ago
  Yeah I took a quick look at their SDK and all the .h and .hpp files say "Confidential and Proprietary". They seemed to provide some kind of tool that converts your model into their own format. Perhaps this could be run with a binary driver without infecting your own stuff? Not that different than certain GPUs I suppose, but I'm going to forget about buying one of those laptops for now.
- verditelabs 2y ago
  The NPU is a Hexagon DSP with HVX, Hexagon Vector Extensions, and HMX, Hexagon Matrix Extensions. The core ISA and vector ISAs are pretty well documented and supported by upstream llvm, but AFAIK HMX is not publicly documented. Core ISA + HVX by itself can probably get you to 1/3 to 1/2 of so of the theoretical peak TOPS for Hexagon. It's been a bit since I've run code on device, but all the support code is in their SDK, and it's easy as pie to get it running on the simulator. QCOM have said that up to ~13B LLMs will run at reasonable, and I think that's a pretty good peak for ~40TOPs and ~150GB/s bandwidth.
SushiHippie 2y ago
https://www.qualcomm.com/developer/blog/2024/05/upstreaming-linux-kernel-support-for-the-snapdragon-x-elite https://www.qualcomm.com/developer/blog/2024/05/upstreaming-... > For kernel versions 6.10 and 6.11, we’re also working on merging more features in these areas: - USB host - On-board display (eDP) - GPU - Memory DCVS - CPUFreq - Speakers/MIC/Headset - Battery - External DP - Suspend/Resume - Camera - Video https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.10 https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.10 Ctrl-F "qualcomm" shows 145 matches, lets hope that some of this stuff lands in 6.10 and 6.11
gregors 2y ago
I'm extremely interested in progress on this front. I'm a person who would definitely purchase an X Elite to run linux ....but only if I could get it installed with minimal effort.

19 points by cl3misch 2y ago

tl;dr: * Not supported out-of-the-box by distros. * Device support is not complete, some models are better than others. * He had to look at websites and mailing lists to get it working.

daghamm 2y ago
Well, these devices where just released, there is no official or even non-official support yet. But I think in 3 or 6 months things will look very different.

Havoc 2y ago

Linux is a good start but I’m really interested in when someone manages to accept the npu with an open stack. ie will a smallish model say 9B or so run at reasonable pace

marshray 2y ago
Yeah I took a quick look at their SDK and all the .h and .hpp files say "Confidential and Proprietary". They seemed to provide some kind of tool that converts your model into their own format. Perhaps this could be run with a binary driver without infecting your own stuff? Not that different than certain GPUs I suppose, but I'm going to forget about buying one of those laptops for now.
verditelabs 2y ago
The NPU is a Hexagon DSP with HVX, Hexagon Vector Extensions, and HMX, Hexagon Matrix Extensions. The core ISA and vector ISAs are pretty well documented and supported by upstream llvm, but AFAIK HMX is not publicly documented. Core ISA + HVX by itself can probably get you to 1/3 to 1/2 of so of the theoretical peak TOPS for Hexagon. It's been a bit since I've run code on device, but all the support code is in their SDK, and it's easy as pie to get it running on the simulator. QCOM have said that up to ~13B LLMs will run at reasonable, and I think that's a pretty good peak for ~40TOPs and ~150GB/s bandwidth.

SushiHippie 2y ago

https://www.qualcomm.com/developer/blog/2024/05/upstreaming-linux-kernel-support-for-the-snapdragon-x-elite https://www.qualcomm.com/developer/blog/2024/05/upstreaming-... > For kernel versions 6.10 and 6.11, we’re also working on merging more features in these areas: - USB host - On-board display (eDP) - GPU - Memory DCVS - CPUFreq - Speakers/MIC/Headset - Battery - External DP - Suspend/Resume - Camera - Video https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.10 https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.10 Ctrl-F "qualcomm" shows 145 matches, lets hope that some of this stuff lands in 6.10 and 6.11

gregors 2y ago

I'm extremely interested in progress on this front. I'm a person who would definitely purchase an X Elite to run linux ....but only if I could get it installed with minimal effort.