6 ms·
They also have a turnkey product with 256 of these things. 1 exaflop + 144TB memory https://nvidianews.nvidia.com/news/nvidia-announces-dgx-gh200-ai-supercomp
by kristopolous 3y ago
They also have a turnkey product with 256 of these things.
1 exaflop + 144TB memory
https://nvidianews.nvidia.com/news/nvidia-announces-dgx-gh200-ai-supercomputer https://nvidianews.nvidia.com/news/nvidia-announces-dgx-gh20...
- bigyikes 3y agoI watched Jensen’s announcement for this. He calls it the worlds largest GPU. It’s just one, giant compute unit. Unlike super computers, which are highly distributed, Nvidia says this is 140 TERABYTES of UNIFIED MEMORY. My mind still gets blown just thinking about it. My poor desktop GPU has 4 gigabytes of memory. Heck, it only has 2 terabytes of storage!
- markus_zhang 3y agoIs it similar to the mainframe in concept?
- tiberious726 3y agoNo, kinda the opposite: mainframes use a litany of sophisticated and special purpose hardware to achieve their tasks (eg hardware io _channels_), while this is a massively overgrown instance of a single kind of hardware (vector processor).
- bitwize 3y agoThe thing that made the lightbulb go off in my head w.r.t. mainframes was understanding that mainframe I/O channels are computers. The mainframe had several dedicated computers that each specifically handled I/O to a terminal, printer, disk or tape drive, punchcard reader, etc. Made I/O programming a breeze, as you just had to tell the channel to read or write, specifying a block of memory to use as a buffer, and the channel would DMA out the data to be written, or DMA in read-in data. It also explains the reason why despite having middling CPU power, mainframes had a reputation for stinkloads of I/O bandwidth so they could process everyone's credit card transactions, airline bookings, and that: the mainframe's CPU was involved very little in I/O, that was all handled by the channel processors!
- deleted 3y ago[deleted]
- coolspot 3y agoIt may be presented as seamless unified memory, but it isn’t. Underlying framework still has to figure out how to allocate your data to minimize cross-unit talk. Each unit has independent CPU, GPU and (V)RAM, but units are interconnected via very fast network.
- westurner 3y agoHow does this compare to HBM High Bandwidth Memory (and GDDR5)? https://en.wikipedia.org/wiki/High_Bandwidth_Memory https://en.wikipedia.org/wiki/High_Bandwidth_Memory
- westurner 3y agoFrom https://news.ycombinator.com/item?id=36211785 https://news.ycombinator.com/item?id=36211785 : > EDIT: found answer to my own question in the datasheet: "The NVIDIA Grace CPU combines 72 Neoverse V2 Armv9 cores with up to 480GB of server-class LPDDR5X memory with ECC." So, this is not stacked RAM like HBM, it's LPDDR5X which a quick search says is 8.5Gbps.
- sliken 3y agoWhat's old is new again. This is basically an updated arm version of the itanium based SGI Altix. Keep in mind unified does not mean uniform, the ram is distributed across all the GPUs.
- bushbaba 3y agoThere’s usecases beyond just ML. Sap Hana could theoretically run on this with greater performance. Same goes for a database. Scaling vertically solves a lot of challenges with distributed ledgers.
- XCSme 3y agoI think distributed computing will go away soon. As computers become more powerful, the cost of "distributing" and transferring the data would be more than simply executing everything locally. Yes, you can still split the task amongst different nodes, or give different problems to different nodes, but the use-case would mostly be solving distinct problems on each note, that splitting the same task across multiple computers. Also, with quantum computers, the parallelization/"distribution" of tasks will be done within the same machine, as it can try all solutions and the same time without having to do divide-et-impera algorithms. Also, in the future, the algorithms will be a lot simpler, and just have FPGA-s like AI chips, where there is no software, the model is directly modelled in the hardware, so each computation is instant (just the time it takes to propagate the electrons or light through the circuit).
- jasonjayr 3y agoI had to look twice at that image, I thought it was a 2 rack-unit device, But, no, it's 24 full 42U racks!!
- kristopolous 3y agoit's just a rendering. I presume nvidia wouldn't be announcing something that they haven't made and confirmed, I wonder why they chose that image. Is it just they haven't done the molding of a production installation? Is it possible that their internal instances might not be that presentable?
- RosanaAnaDana 3y agoI want to see Linus play Doom Eternal on it.
- ChuckNorris89 3y agoIs Crysis no longer a thing?
- all2 3y agoCrysis is a problem because it is single threaded. That's why it was so hard on computers back in the day.
- skhr0680 3y agoBeing single-threaded is why the original Crysis is hard on computers now(!!). When Crysis was being developed people still thought 5-10Ghz was coming Any Day Now.
- all2 3y agoYou might could hit 5Ghz by overclocking the appropriate CPU. I remember 10 years ago looking at the liquid cooling setups needed to hit 5Ghz on a core.
- thomastjeffery 3y agoStock turbo on a 13th gen i5 (performance cores) is 5.1ghz. All you need is good cooling and a stable power supply.
- LoganDark 3y agoMy 12400F boosts to 5GHz for the majority of each day. It's literally normal for me. I've gotten it to almost 5.3GHz before, but it's not really stable above 5.2GHz.
- dtech 3y ago
- MichaelZuo 3y agoThat could probably all fit in a single semi-trailer. It's amazing how dense computation is getting.
- jwr 3y agoAs someone who lived through the first wave of supercomputers (I worked with Cray Y-MP models), it makes me very happy to see the second wave. For a while I thought supercomputing was dead and we would just be connecting lots of PCs with a network and calling that "supercomputers". I still remember how my mind was blown when I first learned that all of the memory in a Cray Y-MP was static RAM. Transistor-based flip-flops: extremely power hungry, but also very fast. Another way of looking at it is that all of its RAM was what we call "cache". This, finally, looks like a supercomputer.
- com2kid 3y agoSRAM is so stupid fun to play with. All of a sudden you don't care so much about the inefficiencies of walking linked lists or trees. When everything is "already in cache", you can worry less about cache efficient algorithms! 1 cycle memory access latency is one of the reasons why tiny embedded MCUs can do things with a fraction of the MHZ of their larger counterparts. Now days of course it is all about tons of memory, tons of bandwidth, craptons of compute, and planning the flow of data ahead of time.
- boringuser2 3y agoOnly within a very small domain was this ever reasonable.
- jwr 3y agoBut that's why there's a "super" in "supercomputers"! Cray loaded a ton of static memory into their computers, then liquid-cooled the whole thing. Sure, the power requirements were through the roof, and you had a whole huge chiller system which you had to run and hope it doesn't fail. If it did fail, you really wanted to shut the machine down fast. From what I recall there was also an emergency propeller inside the back case of the Y-MP 2E, and yes propeller is a much better name for this thing than a "fan". It would delay the inevitable, although dumping those tens of kW of heat into your server room was not something you ever wanted to do. The whole point of all this was that you could do things that you couldn't with "normal" computers. That's why those were called "supercomputers". And I'm so glad that after a hiatus of about 30 years we're getting another wave of exceptional machines, which aren't just bigger PCs.
- coherentpony 3y ago> 1 exaflop To be clear, this is floating point quarter-precision operations when using the FP8 tensor core arithmetic unit [1]. [1] https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-superchip https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-su...
- pezezin 3y agoCame here to post exactly the same link. Not just that, but 1 exaflops of sparse FP8. In comparison, Frontier is 1 exaflops of dense FP64. Try to run this Nvidia system as dense FP64, and it performance will degrade two orders of magnitude. Don't get me wrong, the machine is really impressive, but the advertisement is quite misleading.
- mirekrusin 3y agoOh my, time to upgrade my Pi 4 Model B.
- weinzierl 3y agoIf you upgrade to a Jetson you get GPU power and you can keep the form factor, win - win.
- fennecfoxy 3y agoDoesn't this basically shoot up the list of the TOP500 then? Wonder if they offer a >256 they could be top of the list, easy.
- mk_stjames 3y agoTOP500 uses FP64 performance in ranking. nVidia's 1 exaflop claim is the ~4 petaflops of FP8/INT8 * 256. FP64 performance of modern nvidia gpu's is actually far, far less. The ratio to FP32 isn't even 2:1 anymore (not since Pascal I think) since they realize most machine learning is done with FP32 or less. 64 bit (or 'double precision') is still king in the HPC world though, as it is what you will find in large numerical solutions in fields like computation fluid dynamics, nuclear physics, etc.
- fennecfoxy 3y agoAh fair, I should've known. I suppose the precision is still required for scientific purposes. Thankfully ML stuff now gets more appropriate precision for a speed increase.