3 ms·
Hm... The unified memory model doesn't mean it's faster for inference. Nvidia's GPUs run circles around Apple Silicon in training and inference right now. Thou
by senttoschool 4y ago
Hm...
The unified memory model doesn't mean it's faster for inference. Nvidia's GPUs run circles around Apple Silicon in training and inference right now. Though a part of that is software optimization.
If anything, the reason to have unified memory SoC is to increase general efficiency, not to provide faster AI inference.
I think chip makers will instead start making their NPU (such as Apple's neural engine) really massive. It'll start to take more transistors away from the CPU and GPU. By 2033, perhaps you're buying an NPU with a CPU and GPU attached to it, not the other way around like today.
Also, 2/4/16TB of RAM will not have a 10x price reduction in 10 years. As far as I know, there's nothing in the horizon to do this. Prices will go down but it might stay relatively flat for the next 10 years.
See this chart for $/GB: https://aiimpacts.org/wp-content/uploads/2020/04/memorytrend.png https://aiimpacts.org/wp-content/uploads/2020/04/memorytrend...
Memory stopped getting exponentially cheaper some time around 2011. This is why the MacBook Air still starts with 8GB of RAM.
The one thing that maybe could have helped was Intel's Optane tech, which they abandoned last year. It's memory that sits in between the speed of DRAM and SSDs.
- notfed 4y agoEh, I'm not sure Optane was any faster than modern m.2 SSDs, was it?
- senttoschool 4y agoIntel stopped innovating Optaine years ago. If Intel kept improving it, it would be much faster than SSDs today. Even today, an Optane drive is much faster in latency and important metrics like random reads.
- ly3xqhl8g9 4y ago"an NPU with a CPU and GPU attached to it" Sure, plenty of room for optimizations. "there's nothing in the horizon to do this" Sure, the prediction is that a new curve is laying just a bit beyond the horizon. MacBook Air still starts with 8GB of RAM for the same reason they still start with a 256 GB disk and for the same reason an 8 TB SSD from Apple means +$2,200†: corporate greed since it's good enough. The argument is it won't be good enough in the near future. † whereas a Samsung 870 8TB SSD is around 700 USD