3 ms·
Still? Really? Why?
by MangoToupe 1y ago
Still? Really? Why?
- baby_souffle 1y agoInertia. Almost everybody else was asleep at the wheel for the last decade and you do not catch up to that kind of sustained investment overnight.
- embedding-shape 1y agoFor how shit it all is, it's still the easiest to use, with most available resources when you inevitable need to dig through stuff. Just things like nsight GUI and available debugging options ends up bringing together a better developer experience compared to other ecosystems. I do hope the competitors get better though because the current de facto monopoly helps no-one.
- whywhywhywhy 1y agoBetter support than MPS and nothing Apple is shipping today can compete with even the high end consumer CUDA devices in actual speed.
- MangoToupe 1y agoPresumably the second point is irrelevant if you're choosing among devices with unified memory.
- bigyabai 1y agoIt is not. Unified memory is not a panacea, it says nothing about the compute performance of the hardware. The Spark's GPU gets ~4x the FP16 compute performance of an M3 Ultra GPU on less than half the Mac Studio's total TDP.
- MangoToupe 1y agoright, but that doesn't describe a "high end consumer CUDA device". Nothing under that description has unified memory.
- bigyabai 1y agoEvery CUDA-compatible GPU has had support for unified memory since 2014: https://developer.nvidia.com/blog/unified-memory-cuda-beginners/ https://developer.nvidia.com/blog/unified-memory-cuda-beginn... Can you be a bit more specific what technology you're actually referring to? "Unified memory" is just a marketing term, you could mean unified address space, dual-use memory controllers, SOC integration or Northbridge coprocessors. All are technologies that Nvidia has shipped in consumer products at one point or another, though (Nintendo Switch, Tegra Infotainment, 200X MacBook to name a few).
- nl 1y agoThey mean the ability to run a large model entirely on the GPU without paging it out of a separate memory system.
- whywhywhywhy 1y agoDepends if you care how fast the result arrives. Imagery gen is a very different tool at <12 seconds an image vs nearer to 1 minute.