3 ms·
Ask HN: ARM workstations for homelab CPU based AI Inference testing?
I am eyeing @AmpereComputing's Ampere Altra.
There's a docker image for optimized PyTorch
https://hub.docker.com/r/amperecomputingai/pytorch
I am pretty sure it is just OpenBlas compiled with special options + PyTorch using that, but who knows.
There doesn't seem to be a ton of affordable options. I see the Altra has ASIMD\NEON but not SVE.
If I have a budget of ~3000, is one of their systems probably the best choice? There are several standard formfactor motherboards and even combo kits available at normal outlets.
I'm not _MARRIED_ to ARM, but ~60w idle, 200w tdp when it's busy - that can be cooled quietly, and performance looks decent.
- wmf 3y agoThreadripper 7000 or a Mac Studio probably offers more performance per dollar and per watt.
- dylanetaft 3y agoHPL Linpack was tested on a Q64-22 CPU ~650 Gflops https://github.com/geerlingguy/top500-benchmark/issues/19 https://github.com/geerlingguy/top500-benchmark/issues/19 Memory bandwidth, it's 8 channels of DDR4 on the Ampere vs 7960x 4 channels of DDR5. AMD likely has GFLOPS in the low 1000s if I had to guess, I cannot find any benchmarks of the current gen CPUs, only last gen. I think the Threadripper is actually more expensive, it seems to be a bit less power efficient. Probably similar memory bandwidth due to higher ddr5 clock, faster at math. The PRO threadrippers are even more. It seems competitive, at least, price and performance.
- runjake 3y agoJeff Geerling probably has the most public experience with Ampere, check out his blog[1] and YouTube channel[2]. Personally, I'd use a high-end Apple M3 box or an expensive x64 rig. 1. https://www.jeffgeerling.com/blog https://www.jeffgeerling.com/blog 2. https://www.youtube.com/@JeffGeerling https://www.youtube.com/@JeffGeerling