3 ms·
This is my guess, but does higher end hardware they sell, like the server rack stuff for AI, perhaps have the unified memory? I know standard GPUs don’t. The
by MBCook 1y ago
This is my guess, but does higher end hardware they sell, like the server rack stuff for AI, perhaps have the unified memory?
I know standard GPUs don’t.
The patch suggested one of the reasons for it was to make it easy to develop on a Mac and run on a super computer. So the hardware with the unified memory might be in that class.
- Y_Y 1y agoThe servers don't, but the Jetsons do
- ajuhasz 1y agoThe Jetsons[1] have unified memory[2]. [1] https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/ https://www.nvidia.com/en-us/autonomous-machines/embedded-sy... [2] https://www.nvidia.com/en-us/on-demand/session/gtcspring22-se2600/ https://www.nvidia.com/en-us/on-demand/session/gtcspring22-s...
- tonyarkles 1y agoThey sure do and it's pretty amazing. One iteration of a vision system I worked on got frames from a camera over a Mellanox NIC that supports RDMA (Rivermax), preprocessed the images using CUDA, did inference on them with TensorRT, and the first time a single byte of the inference pipeline hit the CPU itself was when we were consuming the output.
- patrickkrusiec 1y agoThe physical memory is not be unified, but on modern rack scale Nvidia systems, like Grace Hopper or NVL72, the CPU and the GPU(s) share the same virtual address space and have non-uniform memory access to each other's memory.
- freeone3000 1y agoStandard GPUs absolutely do. Since CUDA 11, all CUDA cards expose the same featureset at differing speeds (based on backing capability). You can absolutely (try to) run CUDA UMA on your 2060, and it will complete the computation.