3 ms·

It is not. Unified memory is not a panacea, it says nothing about the compute performance of the hardware. The Spark's GPU gets ~4x the FP16 compute performanc

It is not. Unified memory is not a panacea, it says nothing about the compute performance of the hardware. The Spark's GPU gets ~4x the FP16 compute performance of an M3 Ultra GPU on less than half the Mac Studio's total TDP.

MangoToupe 1y ago
right, but that doesn't describe a "high end consumer CUDA device". Nothing under that description has unified memory.
- bigyabai 1y ago
  Every CUDA-compatible GPU has had support for unified memory since 2014: https://developer.nvidia.com/blog/unified-memory-cuda-beginners/ https://developer.nvidia.com/blog/unified-memory-cuda-beginn... Can you be a bit more specific what technology you're actually referring to? "Unified memory" is just a marketing term, you could mean unified address space, dual-use memory controllers, SOC integration or Northbridge coprocessors. All are technologies that Nvidia has shipped in consumer products at one point or another, though (Nintendo Switch, Tegra Infotainment, 200X MacBook to name a few).
  - nl 1y ago
    They mean the ability to run a large model entirely on the GPU without paging it out of a separate memory system.
    - bigyabai 1y ago
      They're basically describing the Jetson and Tegra lineup, then. Those were featured in several high-end consumer devices, like smart-cars and the Nintendo Switch.
      - nl 1y ago
        Sure but neither had enough memory to be useful for large LLMs. And neither were really consumer offerings.