3 ms·
If Moore's Law had fully kicked over twice more we'd all have 64GB GPUs, enthusiasts would have 2x64GB, and data center build outs wouldn't be needed. Eventual
by com2kid 8mo ago
If Moore's Law had fully kicked over twice more we'd all have 64GB GPUs, enthusiasts would have 2x64GB, and data center build outs wouldn't be needed.
Eventually GPU memory is going to creep up and local models will powerful enough.
- Nevermark 8mo agoI agree. I also think we have only hit the surface of model efficiencies. Apple's M3 Ultra with RAM up to 512GB shared directly across CPU/GPU/NPUs is a great example of an architecture already optimized for local models. I expect Apple will start offering larger RAM sizes for other form factors too. And prices for RAM will drop eventually, because of the extreme demand for RAM with higher densities.
- com2kid 8mo agoIt reminds me of the huge infra investments in Sun and Cisco during the first .com boom, and then 5-10 years later those fancy Sun boxes were out performed by Grandma's Windows XP box.