3 ms·
Nvidia Grace CPU
- peter_d_sherman 5y ago>"AI models are exploding in complexity and size as they improve conversational AI with hundreds of billions of parameters, enhance deep recommender systems with embedding tables of tens of terabytes of data, and enable new scientific discoveries. These massive models are pushing the limits of today's systems. Continuing to scale them for accuracy and usefulness requires fast access to a large pool of memory and a tight coupling of the CPU and GPU." [...] >"Solving the largest AI and HPC problems requires both high-capacity and high-bandwidth memory (HBM). The fourth-generation NVIDIA® NVLink® delivers 900 gigabytes per second (GB/s) of bidirectional bandwidth between the NVIDIA Grace CPU and NVIDIA GPUs. The connection provides a unified, cache-coherent memory address space that combines system and HBM GPU memory for simplified programmability. This coherent, high-bandwidth connection between CPU and GPUs is key to accelerating tomorrow’s most complex AI and HPC problems. New High-Bandwidth Memory Subsystem Using LPDDR5x with ECC Memory bandwidth is a critical factor in server performance, and standard double data rate (DDR) memory consumes a significant portion of overall socket power. The NVIDIA Grace CPU is the first server CPU to harness LPDDR5x memory with server-class reliability through mechanisms like error-correcting code (ECC) to meet the demands of the data center while delivering 2X the memory bandwidth and up to 10X better energy efficiency compared to today’s server memory. The NVIDIA Grace LPDDR5x solution coupled with the large, high-performance, last-level cache delivers the bandwidth necessary for large models while reducing system power to maximize performance for the next generation of workloads."