3 ms·
I hope the LLM benchmarks announced were reasonable and not something gross like using a model that doesn't swap on the Strix Halo, but does on a 4090. HP anno
by sliken 2y ago
I hope the LLM benchmarks announced were reasonable and not something gross like using a model that doesn't swap on the Strix Halo, but does on a 4090.
HP announced a HP Z2 mini g1a, which is bigger than a NUC, but I believe still considered a SFF:
https://www.pcworld.com/article/2567865/hp-z2-mini-g1a-packs-incredible-power-into-a-mini-pc.html https://www.pcworld.com/article/2567865/hp-z2-mini-g1a-packs...
- wmf 2y agoIt's gotta be swapping or something. The 4090 is faster (and far more expensive) than Strix Halo in every way.
- sliken 2y agoAfter digging it looks like 70B Q4 requires 35GB or so. So yes the AMD comparison is a strix halo not swapping vs a 4090 that is swapping. Seems kinda misleading, but is a real advantage for larger models.