3 ms·

I have a Strix Halo device, and like it, but at this price, might as well buy the Nvidia-based ASUS GX 10, if you're buying it for AI. CUDA remains the stronger

I have a Strix Halo device, and like it, but at this price, might as well buy the Nvidia-based ASUS GX 10, if you're buying it for AI. CUDA remains the stronger ecosystem. The AMD is a better desktop machine, as it has a better CPU, but the Nvidia will be a little faster and a little better supported for inference and training workloads. You can almost always do the same things with ROCm, but you're going to work a little more. Though, I will say that Nvidia ships a dogshit custom Ubuntu on their hardware that's hard to deal with. Nvidia is not good at software. I keep thinking they'll get better at it, but I've been dealing with their Jetson line for a couple of years now, and it still sucks. Still a clumsy custom Ubuntu, and it's not as easy as simply installing a different Linux version as it's a complicated image-based thing and no UEFI. At least, I assume they ship Ubuntu on the big devices; I've only dealt with the little embedded Jetson machines. The AMD stuff, being a regular x86_64 PC, you can install pretty much any Linux. I immediately put Fedora on mine.

cmrdporcupine 4mo ago
I ... don't find the Ubuntu on my Spark to be dogshit? It's ... fine? It's just Ubuntu. Hasn't given me any grief and it's so far the only vendor I've seen that actually ships a properly supported Linux on an ARM64 device for Linux, so there's that. I use my ASUS GX10 as my daily driver, my primary workstation. Only thing that doesn't work for me on it is Spotify (probably some DRM thing). Oh, and there's no Signal ARM64 client, it seems. The big advantage of the DGX Spark over the Strix Halo is much faster prefill. Like 5x the speed. Also the networking hardware on it is insanely powerful, though I and 99% of other Spark users, are unlikely to use it to its full capacity.
- SwellJoe 4mo ago
  The unusual bootloader and custom hardware makes modifying and upgrading the OS a challenge. I work on robots that need a custom OS image loaded on the machine. The Nvidia Ubuntu version makes that a huge pain in the ass. They've got binary-only drivers that have to be there, the install/upgrade process is finicky and prone to failure (always recoverable, so far, but all the techs in production I work with have a hard time with it and have to be walked through it, as it's just so alien to folks who work with regular PCs most of the time). Let's just say if I had my druthers, I would not choose Ubuntu, and I really wouldn't choose the Nvidia/ARM spin of Ubuntu. The Strix Halo has the benefit of being entirely a normal x86_64 PC that happens to also have a big chunk of unified memory. You can put pretty much anything on it. Any Linux, regular Windows, probably even a BSD (though good luck getting AI stuff working there). But, as I said, if you're spending $4000 exclusively for inference and AI workloads, you might as well get the Nvidia-based unit. It is better for that.
  - my123 4mo ago
    No the Spark uses UEFI + ACPI Regular Ubuntu or Fedora ISOs do work out of the box Just needs the Nvidia GPU driver install afterwards (And the realtek 10gbe module oot, or blocklist if not used)
    - my123 4mo ago
      And for the Jetsons btw still uses device tree with custom kernels but at least it's UEFI from Orin onwards. For Jetson Orin and later: You can download an ISO from https://developer.nvidia.com/embedded/jetpack/downloads https://developer.nvidia.com/embedded/jetpack/downloads and that'll work. Other distributions are still a bit of a mess though but Yocto is supported now.
    - SwellJoe 4mo ago
      Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.
      - verdverm 4mo ago
        Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)
        cmrdporcupine 4mo ago
        I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways? I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and... ... that was more the disappointing part.

cmrdporcupine 4mo ago

I ... don't find the Ubuntu on my Spark to be dogshit? It's ... fine? It's just Ubuntu. Hasn't given me any grief and it's so far the only vendor I've seen that actually ships a properly supported Linux on an ARM64 device for Linux, so there's that. I use my ASUS GX10 as my daily driver, my primary workstation. Only thing that doesn't work for me on it is Spotify (probably some DRM thing). Oh, and there's no Signal ARM64 client, it seems. The big advantage of the DGX Spark over the Strix Halo is much faster prefill. Like 5x the speed. Also the networking hardware on it is insanely powerful, though I and 99% of other Spark users, are unlikely to use it to its full capacity.

SwellJoe 4mo ago
The unusual bootloader and custom hardware makes modifying and upgrading the OS a challenge. I work on robots that need a custom OS image loaded on the machine. The Nvidia Ubuntu version makes that a huge pain in the ass. They've got binary-only drivers that have to be there, the install/upgrade process is finicky and prone to failure (always recoverable, so far, but all the techs in production I work with have a hard time with it and have to be walked through it, as it's just so alien to folks who work with regular PCs most of the time). Let's just say if I had my druthers, I would not choose Ubuntu, and I really wouldn't choose the Nvidia/ARM spin of Ubuntu. The Strix Halo has the benefit of being entirely a normal x86_64 PC that happens to also have a big chunk of unified memory. You can put pretty much anything on it. Any Linux, regular Windows, probably even a BSD (though good luck getting AI stuff working there). But, as I said, if you're spending $4000 exclusively for inference and AI workloads, you might as well get the Nvidia-based unit. It is better for that.
- my123 4mo ago
  No the Spark uses UEFI + ACPI Regular Ubuntu or Fedora ISOs do work out of the box Just needs the Nvidia GPU driver install afterwards (And the realtek 10gbe module oot, or blocklist if not used)
  - my123 4mo ago
    And for the Jetsons btw still uses device tree with custom kernels but at least it's UEFI from Orin onwards. For Jetson Orin and later: You can download an ISO from https://developer.nvidia.com/embedded/jetpack/downloads https://developer.nvidia.com/embedded/jetpack/downloads and that'll work. Other distributions are still a bit of a mess though but Yocto is supported now.
  - SwellJoe 4mo ago
    Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.
    - verdverm 4mo ago
      Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)
      - cmrdporcupine 4mo ago
        I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways? I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and... ... that was more the disappointing part.