4 ms·

> isnt available in the west [...] Can anyone explain the allure of the Nvidia box The first part might answer the second one. Otherwise, the lack of CUDA and

> isnt available in the west [...] Can anyone explain the allure of the Nvidia box The first part might answer the second one. Otherwise, the lack of CUDA and the nvidia ecosystem of tooling could also explain why it doesn't seem so interesting for AI tasks.

jubilee33 3mo ago
But strix halo boxes themselves are available, just not that one. And despite my concerns about what's said about Cuda and ROCm I have never had a problem running any model, for image or text or voice, the community has done great work in making things work. So the point of the question stands. It's also interesting that the most high end Chinese equipment, both prosumer things like these boxes but also the Huawei professional stack is just not available in the places it would be most appreciated. Not sure if thats china tit for tat, or western "we don't want your commie hardware anyways" But for a lot of people it sucks cause nobody should be paying 4.2k for this product. The value isn't there.
- embedding-shape 3mo ago
  > I have never had a problem running any model, for image or text or voice, the community has done great work in making things work There is a whole world of other tooling and stuff that isn't just for hobbyists to run inference with ML models, but also how to do profiling, debugging and gathering data when you run distributed workloads, and so on. The nsight toolkit seems miles ahead of the competition on other platforms, as just one example.
  - mycall 3mo ago
    Also, you can use handheld PC 395+ 128GB with battery for (semi-){0,1}offline use cases, works great for C2 of cognitive SDR radio (among plenty of other use cases).
icedchai 3mo ago
Yep, stuff is more likely to "just work" on NVidia. Example: pytorch In most benchmarks, the Spark is also faster at the prompt processing / prefill phase.

4 ms·

> isnt available in the west [...] Can anyone explain the allure of the Nvidia box The first part might answer the second one. Otherwise, the lack of CUDA and

by embedding-shape 3mo ago

jubilee33 3mo ago
But strix halo boxes themselves are available, just not that one. And despite my concerns about what's said about Cuda and ROCm I have never had a problem running any model, for image or text or voice, the community has done great work in making things work. So the point of the question stands. It's also interesting that the most high end Chinese equipment, both prosumer things like these boxes but also the Huawei professional stack is just not available in the places it would be most appreciated. Not sure if thats china tit for tat, or western "we don't want your commie hardware anyways" But for a lot of people it sucks cause nobody should be paying 4.2k for this product. The value isn't there.
- embedding-shape 3mo ago
  > I have never had a problem running any model, for image or text or voice, the community has done great work in making things work There is a whole world of other tooling and stuff that isn't just for hobbyists to run inference with ML models, but also how to do profiling, debugging and gathering data when you run distributed workloads, and so on. The nsight toolkit seems miles ahead of the competition on other platforms, as just one example.
  - mycall 3mo ago
    Also, you can use handheld PC 395+ 128GB with battery for (semi-){0,1}offline use cases, works great for C2 of cognitive SDR radio (among plenty of other use cases).
icedchai 3mo ago
Yep, stuff is more likely to "just work" on NVidia. Example: pytorch In most benchmarks, the Spark is also faster at the prompt processing / prefill phase.