3 ms·
Looks like a pretty useful offering, 128Gb Memory Unified, with the ability to be chained. IN the Uk release price looks to be £2999.99 Nice to see AI Inference
by maxbaines 1y ago
Looks like a pretty useful offering, 128Gb Memory Unified, with the ability to be chained. IN the Uk release price looks to be £2999.99 Nice to see AI Inference becoming available to us all, rather than using a GPU ..3090etc.
https://www.scan.co.uk/products/asus-ascent-gx10-desktop-ai-supercomputer-gb10-blackwell-superchip-1tb-ssd-128g-lpddr5x-200gb-connec https://www.scan.co.uk/products/asus-ascent-gx10-desktop-ai-...
- BoredPositron 1y agoI would hold my horses and see if the specs are actually true and not overblown like for the spark otherwise there are better options.
- exasperaited 1y agoAnd if waiting six months is possible, do that. Asus make some really useful things, but the v1 Tinker Board was really a bit problem-ridden, for example. This is similarly way out on the edge of their expertise; I'm not sure I'd buy an out-there Asus v1 product this expensive.
- eightysixfour 1y agoThis is a Spark, so it is not going to be any different.
- atwrk 1y agoAll Sparks only have a memory bandwidth of 270 GB/s though (about the same as the Ryzen AI Max+ 395), while the 3090 has 930 GB/s. (Edit: GB of course, not MB, thanks buildbot)
- buildbot 1y agoI believe you mean GB/s?
- postalrat 1y agoThe 3090 also has 24gb of ram vs 128gb for the spark
- Gracana 1y agoYou'd have to be doing something where the unified memory is specifically necessary, and it's okay that it's slow. If all you want is to run large LLMs slowly, you can do that with split CPU/GPU inference using a normal desktop and a 3090, with the added benefit that a smaller model that fits in the 3090 is going to be blazing fast compared to the same model on the spark.
- Jackson__ 1y agoEh, this is way overblown IMO. The product page claims this is for training, and as long as you crank your batch size high enough you will not run into memory bandwidth constraints. I've finetuned diffusion models streaming from an SSD without noticeable speed penalty at high enough batchsize.
- cmxch 1y agoAt that price (roughly 4000 USD), one could build a full HBM powered Xeon system from the Sapphire Rapids generation. Either build a single socket system and give it some DDR5 to work alongside, or go dual socket and a bit less DDR5 memory.