5 ms·
The Asus Ascent GX10 a Nvidia GB10 Mini PC with 128GB of Memory and 200GbE
- fancyfredbot 2y agoLooks to be a smart play from NVIDIA to recapture the local LLM market, this looks very competitive with Apple M4 and AMD AI Max, but cheaper(!). Nvidia and cheap don't go together most of the time so I think they must have been very worried by developers and enthusiasts buying other hardware. Consistent with their very developer focused strategy and probably a very smart idea since it's low enough spec to avoid canabalising sales of their existing products.
- sliken 2y agoCheaper? The DGX spark is $3,000 with 128GB ram. A framework desktop with the AMD strix halo 395 with 128gb ram is $2,000 and has better memory bandwidth. No price I can see for the identically spec'd (and nearly identical looking) Ascent gx10. [edit] Oops, the Spark is $4,000, only the Ascent is at $3k now. Strix Halo systems vary from slightly slower (6%) to the same (on systems with LPDDR5x-8533, like the HP laptop).
- walterbell 2y agoIncludes RDMA 200GbE NIC. > NVIDIA ConnectX-7 NIC these days often sells for $1500-2200 in single unit quantities, depending on the features and supply of the parts. At $2999 for a system with this buit-in that is awesome.
- kcb 2y agoYea unlike other options this is actually scalable. The question isn't if 1 can outperform the Mac Studio but if 3-4 linked together can.
- Dylan16807 2y agoWhat kind of scaling do you have in mind there? My naive analysis is: A high end Mac should be able to run each layer of an AI task about twice as fast because of the memory bandwidth. And the data going between layers is tiny enough to run over thunderbolt or even normal ethernet. Is there an AI use case that prefers 250GB/s memory bandwidth plus 25GB/s interconnect over 500GB/s memory and 2GB/s interconnect? Are there other major use cases that prefer it?
- fancyfredbot 2y agoUsually the reason you'd want the network bandwidth would be for distributed training. For inference you can probably get by with 2GB/s assuming you can split the layers up nicely. The interconnect can be a bottleneck for inference but only for networks with loads of activations and large batch sizes, or if you are doing tensor level parallelism.
- sliken 2y agoSort of. They come in different speeds, and those prices are for the 400gbe version. The 200gbe, like in the GX10 and spark are more like $1250. Not to mention you have to need that the 200gbe (to cluster 2 of them) and I'd expect the vast majority to buy a single unit, not a pair.
- walterbell 2y agoWith a crossover cable, a single unit could be used for local testing of software that depends on both RDMA and CUDA.
- rubatuga 2y agoAgreed not cheaper by a long shot.
- wtallis 2y agoDouble check your memory bandwidth numbers. AMD's Strix Halo is 256 bits at 8000MT/s for about 238GB/s while NVIDIA's GB10 is 276GB/s (likely 256 bits at the more standard speed of 8533MT/s).
- sliken 2y agoDepends which one, the HP ZBook Ultra uses LPDDR5x-8533, a dead match for what nvidia claims (273GB/sec). Although the DGX spark now costs $4,000 instead of the "starting at $2,999" mentioned a few months ago. So the bandwidth is dead even between AMD and Spark.
- tracker1 2y agoAMD AI Max+ 395 systems should be around $2k with 128gb DDR5X, the Mac Studio maxed out is just north of $12k. So it's in the middle depending. This will also vary greatly in terms of usability for other things. That said, a nice OEM option if you need something for AI workloads where the GPU market is completely soaked with scalpers. Been considering a drive to California just so I can get a GPU directly at MicroCenter instead of paying scalper overheads.
- sliken 2y agoYes, you can spend $12k on a mac studio. However the Mac studio with the m4 max has 128GB ram, double the memory bandwidth, and costs $3,699. LESS than the DGX spark. Granted it doesn't have 200gbe. If it's the memory bandwidth you are after the Mac Mini with the m4 pro has similar, but max 64GB ram.
- fancyfredbot 2y agoArticle is the ASUS system which is $3000, so cheaper than the M4 max.
- sliken 2y agoIndeed, 23% more money for double the memory bandwidth.
- deleted 2y ago[deleted]
- rbanffy 2y ago> This will also vary greatly in terms of usability for other things. This is key. Nvidia has a terrible reputation with long term support (as market leaders, they can easily afford that). Apple just now (last November) dropped OS updates for their 2014 boxes. While a Mac Studio 2025 will not be a ridiculous amount of compute power in 10 years, I fully expect Nvidia to completely abandon support and software updates for this in five years tops. Hopefully, considering the interest it generated, I'd hope the Linux crowd will be able to carry it further, maybe with decent open-source drivers, way past the expiration date Nvidia sets.
- fancyfredbot 2y agoToo late to edit my post but wanted to apologise for causing confusion. I suppose "cheapest" can be a very subjective term if the comparison is between things with different capabilities. NVIDIA is a cheaper option than AMD only if you compare assume you want/need the fast networking NVIDIA are bundling with their system. According to the article the NIC would add $1500-$2000 to the price of another systems. I also failed to account for the extra memory bandwidth offered by M4 max. The apple system costs more but if you want/need that bandwidth then it's the cheapest of the three. I guess "the system has a niche where it offers very good price/performance" is what I should have said. Not as snappy though.
- bitsandboots 2y agoI'd say the SGX is the "cheapest" only if you're trying to go beyond 96GB. the AMD system is 96GB max for the GPU. The 128GB allocates a minimum of 32GB to the CPU. the Nvidia system is designed to be connected to a second if so desired, making it the cheapest way to get 256GB. If you're just going for something under 96GB, haven't seen something cheaper than the AMD system for anything that can't fit on a traditional GPU. And even then, GPUs are obscene ripoff prices lately. Here's hoping these won't be scalped too.
- ezschemi 2y agoThe 96GB is the maximum on Windows. On Linux, you can allocate 110GB.
- RamboRogers 2y agoMemory speed is way slower than mac studio, be interested to see benchmarks.
- bitsandboots 2y agoBenchmarks of what? Memory speed matters for some things but not others. It matters a lot for AI training, but less for AI inference. It matters a lot for games too, but nobody would play a game on this or a mac.
- buildbot 2y agoAI inference is actually typically bandwidth limited compared to training, which can re-use the weights for all tokens <sequence length> * <batch size>. Inference, specifically decoding, requires you to read all of the weights for each new token, so the flops per byte are much lower during inference!
- Tepix 2y agoDo the math. A dense 100GB model will be awfully slow at 273GB/s mem bandwidth.- So far, project digits looks disappointing.
- fancyfredbot 2y agoA 100GB model running on one of these would be a disaster, you'd probably run out of space for the KV cache apart from anything else. But if you had two of them performance should be acceptable. I think they would be canabalising their other product lines if they had more memory bandwidth.
- nashashmi 2y agoThe coolest part is the thing is powered by a USB C. Granted it is not an intel, but the first powerful mini pc that does not have a separate dc power port is a breath of fresh air .
- rbanffy 2y agoIt also means there will be a power brick under the desk. I'm not sure there are monitors that can power it from their USB-C ports.
- deleted 2y ago[deleted]
- zokier 2y agoAt 170 watts? Unlikely that there are many monitors with that level of USB-PD.
- rbanffy 2y agoI wonder if you can attach it to two monitors and power it that way. I don't think anyone expects the user of such a beast to have a single screen.
- anonymousiam 2y agoThere are plenty of docks that will supply 100W via USB-C PD.
- rbanffy 2y agoThis draws 170. Not ridiculous, but quite a lot. And the dock can still count as a power brick.
- simoncion 2y ago> The coolest part is the thing is powered by a USB C. [This is] a breath of fresh air. Serious question, looking for a serious answer from you: Why? The wafer-style USB-C connector is mechanically inferior to the size of barrel-style connectors that are typically used for 100+W applications. It's far less resistant to torque and shear, and due to its small size is significantly easier to accidentally dislodge. While accidental disconnection isn't a huge problem for a battery-powered machine, this thing is wall-powered. Making things worse, unless you want to roll the dice on Amazon not sending you something that might burn your house down due to underspecced conductors and/or connectors, you're going to have to pay a significant fraction of the cost of the PSU for the cable to connect it. There are also the ergonomic factors to consider. On the overwhelming majority of consumer hardware that's powered with a barrel-style connector, there's only one place where that connector fits. However, on much-to-most of the consumer hardware that's powered with a USB-C connector, there are many plugs that will accept the power supply cable but will refuse to use it to power the hardware. Perhaps one day in the distant future, manufacturers will universally pay the money to make all USB-C-shaped plugs capable of powering the USB-C-connector-powered-hardware they're built in to, but that day is not today. And sure, you can reasonably retort with "Well, you just keep trying plugs until it works.", as well as "Well, just turn on a light and read the printing on the plugs.". These aren't huge hassles. But I remember that one of the big hassles that the USB-C connector saved us from was "You don't have to try to plug it in three times to get it right.". Going from "Well, it just works, first try!" to "Well, you just have to find the right plug to make it work. Keep trying!" is backwards and a little sad.
- haswell 2y agoThe Framework Desktop [0] also sits in this space and looks pretty promising. Starts at $1,999 for 128GB RAM and no storage. - [0] https://frame.work/desktop https://frame.work/desktop Edit: I misread the specs and posted this a bit too hastily. I'd delete this comment but it already has children.
- cosmotic 2y agoI think these are totally different spaces. Th OP is for an AI dev machine. Framework Desktop is general purpose.
- rbanffy 2y agoAny computer is general purpose if you want it hard enough. I hope this being more accessible than other Nvidia gear allows it to develop a healthy software ecosystem that doesn't depend so much on Nvidia.
- eckelhesten 2y ago>Any computer is general purpose if you want it hard enough. True, but not any general purpose computer comes with 1000TOPS of computational power.
- sliken 2y ago1000 TOPS sounds good, until you notice the 273GB/sec. This will allow larger models than consumer GPUs, but be rather slow running them.
- jauntywundrkind 2y agoThey both have ~0.25TBps memory bandwidth & huge GPUs. They very much are in the same space. I'm very Team Red (AMD) but I dig & respect the 4x 40Gbps USB ports here. Its not popular but it is a good way to attach pretty fast storage. Also the 200Gbps connect-x7 is just awesome. Users can add what they will to Strix Halo, but it's neat seeing good connectivity on chip. I hope AMD can get better about USB4 & Thunderbolt 4. It really feels like it should be part of their APUs. Apple shipping 6x TB5 ports on the new M3 Ultra is a beautiful example of what life should be like.
- deleted 2y ago[deleted]