4 ms·

I ... don't find the Ubuntu on my Spark to be dogshit? It's ... fine? It's just Ubuntu. Hasn't given me any grief and it's so far the only vendor I've seen th

I ... don't find the Ubuntu on my Spark to be dogshit? It's ... fine? It's just Ubuntu. Hasn't given me any grief and it's so far the only vendor I've seen that actually ships a properly supported Linux on an ARM64 device for Linux, so there's that. I use my ASUS GX10 as my daily driver, my primary workstation. Only thing that doesn't work for me on it is Spotify (probably some DRM thing). Oh, and there's no Signal ARM64 client, it seems. The big advantage of the DGX Spark over the Strix Halo is much faster prefill. Like 5x the speed. Also the networking hardware on it is insanely powerful, though I and 99% of other Spark users, are unlikely to use it to its full capacity.

SwellJoe 4mo ago
The unusual bootloader and custom hardware makes modifying and upgrading the OS a challenge. I work on robots that need a custom OS image loaded on the machine. The Nvidia Ubuntu version makes that a huge pain in the ass. They've got binary-only drivers that have to be there, the install/upgrade process is finicky and prone to failure (always recoverable, so far, but all the techs in production I work with have a hard time with it and have to be walked through it, as it's just so alien to folks who work with regular PCs most of the time). Let's just say if I had my druthers, I would not choose Ubuntu, and I really wouldn't choose the Nvidia/ARM spin of Ubuntu. The Strix Halo has the benefit of being entirely a normal x86_64 PC that happens to also have a big chunk of unified memory. You can put pretty much anything on it. Any Linux, regular Windows, probably even a BSD (though good luck getting AI stuff working there). But, as I said, if you're spending $4000 exclusively for inference and AI workloads, you might as well get the Nvidia-based unit. It is better for that.
- my123 4mo ago
  No the Spark uses UEFI + ACPI Regular Ubuntu or Fedora ISOs do work out of the box Just needs the Nvidia GPU driver install afterwards (And the realtek 10gbe module oot, or blocklist if not used)
  - my123 4mo ago
    And for the Jetsons btw still uses device tree with custom kernels but at least it's UEFI from Orin onwards. For Jetson Orin and later: You can download an ISO from https://developer.nvidia.com/embedded/jetpack/downloads https://developer.nvidia.com/embedded/jetpack/downloads and that'll work. Other distributions are still a bit of a mess though but Yocto is supported now.
  - SwellJoe 4mo ago
    Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.
    - verdverm 4mo ago
      Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)
      - cmrdporcupine 4mo ago
        I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways? I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and... ... that was more the disappointing part.
        verdverm 4mo ago
        The Spark does a system update on first boot, so they would have updated had they been installed I'm running vllm + GoModel on k3s, works like a charm. Even wrote some CUE last night to generate the values file for my Helm chart. It calculates the GPU percentage for me from the memGB I assign to a model.

SwellJoe 4mo ago

The unusual bootloader and custom hardware makes modifying and upgrading the OS a challenge. I work on robots that need a custom OS image loaded on the machine. The Nvidia Ubuntu version makes that a huge pain in the ass. They've got binary-only drivers that have to be there, the install/upgrade process is finicky and prone to failure (always recoverable, so far, but all the techs in production I work with have a hard time with it and have to be walked through it, as it's just so alien to folks who work with regular PCs most of the time). Let's just say if I had my druthers, I would not choose Ubuntu, and I really wouldn't choose the Nvidia/ARM spin of Ubuntu. The Strix Halo has the benefit of being entirely a normal x86_64 PC that happens to also have a big chunk of unified memory. You can put pretty much anything on it. Any Linux, regular Windows, probably even a BSD (though good luck getting AI stuff working there). But, as I said, if you're spending $4000 exclusively for inference and AI workloads, you might as well get the Nvidia-based unit. It is better for that.

my123 4mo ago
No the Spark uses UEFI + ACPI Regular Ubuntu or Fedora ISOs do work out of the box Just needs the Nvidia GPU driver install afterwards (And the realtek 10gbe module oot, or blocklist if not used)
- my123 4mo ago
  And for the Jetsons btw still uses device tree with custom kernels but at least it's UEFI from Orin onwards. For Jetson Orin and later: You can download an ISO from https://developer.nvidia.com/embedded/jetpack/downloads https://developer.nvidia.com/embedded/jetpack/downloads and that'll work. Other distributions are still a bit of a mess though but Yocto is supported now.
- SwellJoe 4mo ago
  Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.
  - verdverm 4mo ago
    Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)
    - cmrdporcupine 4mo ago
      I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways? I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and... ... that was more the disappointing part.
      - verdverm 4mo ago
        The Spark does a system update on first boot, so they would have updated had they been installed I'm running vllm + GoModel on k3s, works like a charm. Even wrote some CUE last night to generate the values file for my Helm chart. It calculates the GPU percentage for me from the memGB I assign to a model.