4 ms·

I watched Jensen’s announcement for this. He calls it the worlds largest GPU. It’s just one, giant compute unit. Unlike super computers, which are highly dist

I watched Jensen’s announcement for this. He calls it the worlds largest GPU. It’s just one, giant compute unit. Unlike super computers, which are highly distributed, Nvidia says this is 140 TERABYTES of UNIFIED MEMORY. My mind still gets blown just thinking about it. My poor desktop GPU has 4 gigabytes of memory. Heck, it only has 2 terabytes of storage!

markus_zhang 3y ago
Is it similar to the mainframe in concept?
- tiberious726 3y ago
  No, kinda the opposite: mainframes use a litany of sophisticated and special purpose hardware to achieve their tasks (eg hardware io _channels_), while this is a massively overgrown instance of a single kind of hardware (vector processor).
  - bitwize 3y ago
    The thing that made the lightbulb go off in my head w.r.t. mainframes was understanding that mainframe I/O channels are computers. The mainframe had several dedicated computers that each specifically handled I/O to a terminal, printer, disk or tape drive, punchcard reader, etc. Made I/O programming a breeze, as you just had to tell the channel to read or write, specifying a block of memory to use as a buffer, and the channel would DMA out the data to be written, or DMA in read-in data. It also explains the reason why despite having middling CPU power, mainframes had a reputation for stinkloads of I/O bandwidth so they could process everyone's credit card transactions, airline bookings, and that: the mainframe's CPU was involved very little in I/O, that was all handled by the channel processors!
deleted 3y ago
[deleted]
coolspot 3y ago
It may be presented as seamless unified memory, but it isn’t. Underlying framework still has to figure out how to allocate your data to minimize cross-unit talk. Each unit has independent CPU, GPU and (V)RAM, but units are interconnected via very fast network.
- westurner 3y ago
  How does this compare to HBM High Bandwidth Memory (and GDDR5)? https://en.wikipedia.org/wiki/High_Bandwidth_Memory https://en.wikipedia.org/wiki/High_Bandwidth_Memory
  - westurner 3y ago
    From https://news.ycombinator.com/item?id=36211785 https://news.ycombinator.com/item?id=36211785 : > EDIT: found answer to my own question in the datasheet: "The NVIDIA Grace CPU combines 72 Neoverse V2 Armv9 cores with up to 480GB of server-class LPDDR5X memory with ECC." So, this is not stacked RAM like HBM, it's LPDDR5X which a quick search says is 8.5Gbps.
sliken 3y ago
What's old is new again. This is basically an updated arm version of the itanium based SGI Altix. Keep in mind unified does not mean uniform, the ram is distributed across all the GPUs.
bushbaba 3y ago
There’s usecases beyond just ML. Sap Hana could theoretically run on this with greater performance. Same goes for a database. Scaling vertically solves a lot of challenges with distributed ledgers.
XCSme 3y ago
I think distributed computing will go away soon. As computers become more powerful, the cost of "distributing" and transferring the data would be more than simply executing everything locally. Yes, you can still split the task amongst different nodes, or give different problems to different nodes, but the use-case would mostly be solving distinct problems on each note, that splitting the same task across multiple computers. Also, with quantum computers, the parallelization/"distribution" of tasks will be done within the same machine, as it can try all solutions and the same time without having to do divide-et-impera algorithms. Also, in the future, the algorithms will be a lot simpler, and just have FPGA-s like AI chips, where there is no software, the model is directly modelled in the hardware, so each computation is instant (just the time it takes to propagate the electrons or light through the circuit).

markus_zhang 3y ago

Is it similar to the mainframe in concept?

tiberious726 3y ago
No, kinda the opposite: mainframes use a litany of sophisticated and special purpose hardware to achieve their tasks (eg hardware io _channels_), while this is a massively overgrown instance of a single kind of hardware (vector processor).
- bitwize 3y ago
  The thing that made the lightbulb go off in my head w.r.t. mainframes was understanding that mainframe I/O channels are computers. The mainframe had several dedicated computers that each specifically handled I/O to a terminal, printer, disk or tape drive, punchcard reader, etc. Made I/O programming a breeze, as you just had to tell the channel to read or write, specifying a block of memory to use as a buffer, and the channel would DMA out the data to be written, or DMA in read-in data. It also explains the reason why despite having middling CPU power, mainframes had a reputation for stinkloads of I/O bandwidth so they could process everyone's credit card transactions, airline bookings, and that: the mainframe's CPU was involved very little in I/O, that was all handled by the channel processors!

deleted 3y ago

[deleted]

coolspot 3y ago

It may be presented as seamless unified memory, but it isn’t. Underlying framework still has to figure out how to allocate your data to minimize cross-unit talk. Each unit has independent CPU, GPU and (V)RAM, but units are interconnected via very fast network.

westurner 3y ago
How does this compare to HBM High Bandwidth Memory (and GDDR5)? https://en.wikipedia.org/wiki/High_Bandwidth_Memory https://en.wikipedia.org/wiki/High_Bandwidth_Memory
- westurner 3y ago
  From https://news.ycombinator.com/item?id=36211785 https://news.ycombinator.com/item?id=36211785 : > EDIT: found answer to my own question in the datasheet: "The NVIDIA Grace CPU combines 72 Neoverse V2 Armv9 cores with up to 480GB of server-class LPDDR5X memory with ECC." So, this is not stacked RAM like HBM, it's LPDDR5X which a quick search says is 8.5Gbps.

sliken 3y ago

What's old is new again. This is basically an updated arm version of the itanium based SGI Altix. Keep in mind unified does not mean uniform, the ram is distributed across all the GPUs.

bushbaba 3y ago

There’s usecases beyond just ML. Sap Hana could theoretically run on this with greater performance. Same goes for a database. Scaling vertically solves a lot of challenges with distributed ledgers.

XCSme 3y ago

I think distributed computing will go away soon. As computers become more powerful, the cost of "distributing" and transferring the data would be more than simply executing everything locally. Yes, you can still split the task amongst different nodes, or give different problems to different nodes, but the use-case would mostly be solving distinct problems on each note, that splitting the same task across multiple computers. Also, with quantum computers, the parallelization/"distribution" of tasks will be done within the same machine, as it can try all solutions and the same time without having to do divide-et-impera algorithms. Also, in the future, the algorithms will be a lot simpler, and just have FPGA-s like AI chips, where there is no software, the model is directly modelled in the hardware, so each computation is instant (just the time it takes to propagate the electrons or light through the circuit).