5 ms·
Nvidia GPU roadmap confirms it: Moore's Law is dead and buried
- deleted 2y ago[deleted]
- rockwotj 2y ago> Anytime Meta, Oracle, Microsoft, or anyone else announces another AI bit barn, a juicy power purchase agreement usually follows. Meta's mega DC being birthed in the bayou was announced alongside a 2.2GW gas generator plant — so much for those sustainability and carbon neutrality pledges. Ouch, I grew up where Meta created its first data center and while it is very efficient (https://engineering.fb.com/2011/04/14/core-infra/designing-a-very-efficient-data-center/ https://engineering.fb.com/2011/04/14/core-infra/designing-a...) I remember a more recent article about how they are zero net emissions (https://tech.facebook.com/engineering/2021/11/10-years-world-class-data-centers/ https://tech.facebook.com/engineering/2021/11/10-years-world...). I wonder if they have kept that in this AI rush (along with most of the datacenter companies). EDIT: a direct link to net zero for Meta: https://tech.facebook.com/ideas/2020/9/facebooks-path-to-net-zero/ https://tech.facebook.com/ideas/2020/9/facebooks-path-to-net...
- xienze 2y ago> so much for those sustainability and carbon neutrality pledges. Can’t they just buy indulgences^W carbon offset credits?
- Jgrubb 2y agolol "carbon indulgences", I'm definitely going to use that from now on.
- infecto 2y agoMaybe an unpopular opinion here, but constraints—especially around energy, are a key driver of innovation. Scarcity forces optimization. Historically, some of the most important advancements have come not from abundance, but from being forced to do more with less.
- piva00 2y agoLimitations breed creativity, it works for almost any endeavour in the arts or engineering.
- gizmo 2y agoIntel made 3.7 GHz CPUs 20 years ago. Modern CPUs run at the same frequency. So was Moore's law dead 20 years ago? Clearly not. Right now Nvidia's focus is on making bigger silicon with faster interconnects. But pretty soon that will stop working and then the focus will shift again. It's still early days for AI. People have only just started working on making dedicated AI chips. Presuming that performance per watt or per mm^2 of silicon cannot go up from here seems silly.
- Shorel 2y ago> So was Moore's law dead 20 years ago? Clearly not. Agree. It is only 'dead' because Moore's law is about the cost per transistor, and Nvidia wants to keep that price constant. Double the transistors? Double the price of the chips. That's Nvidia's wet dream. I hope this backfires on Nvidia and the standard Moore's law returns.
- gizmo 2y agoNo, Moore's law is the number of transistors on a chip doubling every 2 years. Quantity not cost.
- Shorel 2y agoWhat Moore said in the paper was about cost per transistor. It got misinterpreted during the 90s, that's all.
- rsfern 2y agoWhat is “standard Moore’s law”? People say that Moore’s law has been dead for 20 years because that’s when the industry hit the power wall and had to pivot from simply reducing transistor size to parallelism and these other engineering strategies for keeping the performance growth curve going. I don’t think that’s coming back without solving some really difficult materials and semiconductor physics problems
- Shorel 2y ago
- alecco 2y agoThe title is usual clickbaity Register but the content is good enough. TL;DW: * The improvements don't come from transistor density but other tricks like putting 2 chips together, using smaller 4bit format, more on-chip memory, 2x memory bandwidth * And Nvidia is packing more GPUs in the same rack and consuming unheard of amounts of power, with a huge toll on datacenter infra And it seems all these have growth limits. No more 2x every year or even every other year.
- formerly_proven 2y agoHeadline marketing "AI TOPS" is also based on whatever smaller format a new generation supports. E.g. doubling headline marketing performance from Ada to Blackwell by going from FP8 to FP4.
- Y_Y 2y agoDon't forget "structured sparsity" where two out of every four weights must be zero. Another trick that Nvidia still happily assume for the sake of a graph or an headline figure.
- Elextric 2y agoEven if we achieve 2x every 3-4 years, isn't it still massive in absolute ?
- anonfordays 2y agoYes, exponential growth gets quite large quite fast. Think about how a $300 graphics card today competes against one from 15 years ago. They're orders of magnitude faster, more memory, etc.
- deleted 2y ago[deleted]
- logicchains 2y agoThere's a lot of progress on optical computing recently, with some of its proponents suggesting they could get up to 100x speedup on some things like LLMs relative to current chips. If they ever manage to deliver then we'll see a lot of 2x or more improvements.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- gchamonlive 2y agoNo surprise there, that Moore's Law requires a healthy competition environment for it to be likely to take effect and it's been a while since Nvidia had actual competition. Of course it's much more complex than that. The nature of the problems that tech now has to solve is different and as stated in the article, Nvidia hit many roadblocks, but I still think if it had healthy competition, other brands would step in and make it more likely for a creative solution to manifest itself.
- diggan 2y ago> Nvidia hit many roadblocks Roadblocks they've seemed to put on themselves, in the name of market segmentation. There is a huge market sitting between consumer and "professional" GPUs, and nvidia is trying to milk the market as much as they can before any of the competitors get their shit together.
- noosphr 2y agoNVidia only has so much silicon area it can get from its partners. Right now AI datacenter silicon is worth much more than its weight in gold, but consumer grade silicon is worth a fraction of that. I predict we are pretty close to the death of the GPU as a discrete component. We will have TPUs that live in work stations and data centers as discrete components, and integrated graphics cards that live on the CPU die.
- jasode 2y ago> Moore's Law requires a healthy competition environment for it to be likely to take effect and it's been a while since Nvidia had actual competition. Moore's Law was an extrapolation on transistor density which is a manufacturing process in a chip foundry. NVIDIA is a fabless company that depends on physical process improvements from companies like ASML & TSMC. So the competition you're speaking of is happening more at the foundry layer which would be TSMC vs Intel vs Samsung vs China-chip-maker (e.g. SMIC). At the lithography layer, ASML's EUV has no competition from Nikon and Canon.
- 2y ago
- andrewstuart 2y agoNvidia just wants you to accept higher prices for less computing power. That’s why Jensen is always saying Moores Law is dead - so you buy less of his sand for more bucks.
- danw1979 2y agoThe article mentions 600kW power to a single rack might be a thing soon. Surely not air-cooled ? This seems wild to me. I used to warm myself next to 4kW racks of telco gear when I was still doing overnight work (20 ish years ago) and thought this was a lot of power to be using in such a small space…
- infecto 2y agoAnecdote but all of the PR around this release displayed only liquid cooled so I assume so.
- DeathArrow 2y agoThey can use the resulting heat for heating a small town.
- imhoguy 2y agoThey should transfer that heat to steam engines to deliver more power back to DC... /s
- bigfishrunning 2y agoWhy the /s? recapturing waste heat and using it to reduce your energy consumption seems like a good idea if you can get it to work
- imhoguy 2y agoActually you are right, while writing it I have imagined a steam engine in the rack itself :)
- DeathArrow 2y agoMaybe we are near the limits of the silicon. I wonder if changing to some other medium, carbon, optical circuits etc., wouldn't allow us to benefit from Moore's law again.
- beautifulfreak 2y agoMaybe bismuth oxyselenide https://www.tomshardware.com/pc-components/cpus/chinese-university-designed-worlds-first-silicon-free-2d-gaafet-transistor-new-bismuth-based-tech-is-both-the-fastest-and-lowest-power-transistor-yet https://www.tomshardware.com/pc-components/cpus/chinese-univ...
- SirMaster 2y agoI thought we were getting close to the limits of electrons and issues of quantum tunneling into adjacent circuits and such. Would a different material even change that?
- hwers 2y agoIf price per compute keeps going down we effectively keep having moore’s law for parallel compute like GPUs. (Just get better at making ML models that don’t have as big of a communication bottleneck.)
- bhouston 2y agoI have been reading Moore's Law is dead articles for the last 30 years. You sort of become immune to the "sky is falling" narrative after a while. Tech leaders love this narrative: 2005: Intel CEO https://hothardware.com/news/moores-law-is-dead-says-gordon-moore https://hothardware.com/news/moores-law-is-dead-says-gordon-... 2009: Sandisk CEO https://archive.nytimes.com/bits.blogs.nytimes.com/2009/05/22/counting-down-to-the-end-of-moores-law/ https://archive.nytimes.com/bits.blogs.nytimes.com/2009/05/2... 2010: NVIDIA VP https://finance.yahoo.com/news/2010-05-03-nvidia-vp-says-moores-law-is-dead.html https://finance.yahoo.com/news/2010-05-03-nvidia-vp-says-moo... 2016: Intel CEO https://www.nytimes.com/2016/05/05/technology/moores-law-running-out-of-room-tech-looks-for-a-successor.html https://www.nytimes.com/2016/05/05/technology/moores-law-run... 2017: NVIDIA CEO https://www.extremetech.com/cars/256558-nvidias-ceo-declares-moores-law-dead https://www.extremetech.com/cars/256558-nvidias-ceo-declares... 2019: NVIDIA CEO https://www.cnet.com/tech/computing/moores-law-is-dead-nvidias-ceo-jensen-huang-says-at-ces-2019/ https://www.cnet.com/tech/computing/moores-law-is-dead-nvidi... 2022: NVIDIA CEO https://www.barrons.com/articles/nvidia-graphic-card-prices-moores-law-51663778838 https://www.barrons.com/articles/nvidia-graphic-card-prices-... The only time this felt true was on the desktop/Windows/Linux for CPUs when Intel had an uncontested monopoly from 2009 - 2016: https://cdn.arstechnica.net/wp-content/uploads/2020/11/CPU-performance-retrospective.mt-full-range.png https://cdn.arstechnica.net/wp-content/uploads/2020/11/CPU-p... But it was broken by AMD's Ryzen architecture on one side and Apple's M series on the other.
- Tostino 2y agoRegardless of what you feel, it is long dead.
- bhouston 2y agoI look at these benchmarks and I completely disagree: https://browser.geekbench.com/macs/mac-mini-late-2020 https://browser.geekbench.com/macs/mac-mini-late-2020 https://browser.geekbench.com/macs/mac-mini-2024-10c-cpu https://browser.geekbench.com/macs/mac-mini-2024-10c-cpu I am quite satisfied with the rapid speed increases on Apple's CPUs the last couple years.
- fancyfredbot 2y agoThe economist wrote an article "Jensen Huang says Moore’s law is dead. Not quite yet" (https://www.economist.com/science-and-technology/2023/12/13/jensen-huang-says-moores-law-is-dead-not-quite-yet https://www.economist.com/science-and-technology/2023/12/13/...) which refutes this claim quite nicely. It's two years old but still pretty accurate.
- sheepscreek 2y agoTL;DR it’s dead in the sense, we’re falling short of doubling the transistor count every 18-24 months. That said, in terms of horizontal expansion (tripling or quadrupling the number of computational cores), surely the main challenge must be the lack of bandwidth to the memory to power such insane data transfer rates. You could of course dedicate some amount of memory to each compute node, then the problem is one of quickly splitting the workload into chunks and sending them their separate way with no need to move them around. In a way, LLM and neural networks do function that way. Maybe it is time to build hardware that maps 1:1 to the software architecture.
- sheepscreek 2y agoI always like to check my assumptions, here is what ChatGPT said of the above: Yes, there is truth to that argument—both in the limitations it describes and the vision it hints at. Let’s break it down: ⸻ 1. Bandwidth Bottleneck with Horizontal Scaling • True: As you scale out the number of computational cores (SMs in NVIDIA GPUs, for example), the global memory bandwidth becomes a bottleneck unless it scales proportionally. • GDDR6, HBM, and even on-package memory (e.g. Apple’s M-series) are attempts to address this. • Still, there’s a ceiling: latency and contention start dominating. • Problem: Many workloads, especially in ML, are memory-bound rather than compute-bound. Doubling cores doesn’t help if data can’t be fed fast enough. ⸻ 2. Local Memory per Compute Unit • True and Already Happening: Architectures are increasingly moving toward giving compute units local memory: • NVIDIA’s SMs have shared memory (very fast, but small). • AMD’s Infinity Cache, or on-chip memory on M1/M2, is another example. • TPUs and custom ASICs use SRAM close to the compute logic for matrix multiplies. • This is reminiscent of scratchpad memory in SIMD systems or old-school Cell processors. ⸻ 3. Workload Partitioning and Data Locality • Critical Insight: Yes, mapping data locality to compute locality is the key. • LLMs and other neural nets are mostly large tensor operations. • These can be tiled and distributed to cores with minimal inter-core communication if partitioned right. • This is why hardware-software co-design is becoming more important: • NVIDIA’s Tensor Cores and transformer engine • Google’s TPU with XLA compiler • Apple’s use of unified memory for CPU, GPU, and ML cores ⸻ 4. Hardware Matching Software Architectures • Emerging Trend: We are already seeing this: • LLM accelerators (e.g. Groq, Tenstorrent, Cerebras) aim to align hardware with transformer-like workloads. • Neuromorphic chips and systolic arrays are trying to escape the Von Neumann bottleneck by mimicking data flow instead of instruction flow. • Vision: Instead of generic ALUs doing many different things, you have domain-specific architectures (DSAs) optimized for matrix math, attention, etc. ⸻ Conclusion You’re touching on a real and current transition in hardware design: as Moore’s Law slows and workloads become increasingly specialized (especially ML/AI), the bottlenecks shift from FLOPS to memory bandwidth, latency, and data movement. Yes, scaling horizontally is increasingly limited by data movement costs, and yes, future architectures will (and already do) look more like the software they’re meant to run. Would you like a more technical breakdown of how LLM workloads are actually split across GPU memory hierarchies or how specific hardware like Cerebras or TPUs tackle this?
- James_K 2y agoI don't know why, but the idea of a 72 GPU rack provokes a kind of existential dread in me.
- Gravityloss 2y agoAn earlier big data mantra said to move computation to where the data is - as data is too big to move around. Then I guess here, it could turn out, computation should be done where energy is, as energy is potentially even harder to move around than data? Tens of gigabytes per second is easy through one tiny fiber. A bundle is easily laid by an ordinary excavator. In contrast a transmission line for a gigawatt for a few hundred kilometers takes billions. With proper engineering with nothing fundamentally new, one could get proper integrated water cooled racks with very high computing density and power usage. Then put those racks in a building close to a nuclear powerplant (outside the security zone perhaps though). Or a wind or solar park + batteries. Can you circumvent the costs associated with using the public power grid? Extremely simplified, you only need fiber in and out of this power+computing facility. Ultimately you could do this in space with solar power and laser up/downlink. Cooling might be problematic though. France at least used to have a dedicated nuclear plant just for uranium enrichment. Before electricity, a lot of industry centered to places near running water for sawmills, flour mills, metal hammering and so on. With electricity these were in many cases decoupled. But things that get energy intense enough and don't need material transport or local labor, it might make sense to locate close to energy sources again.
- seec 2y agoI think this is already happening. OVH built a datacenter in Canada, close to a big dam, so they could get access to cheap electricity and a lot of it. This is just one example, but I would be surprised if that wasn't a consideration in more datacenter placements. The problem of course is to find a place where electricity is abundant, but you also have easy access to the large fiber backbone network. If you have a lot of traffic, fibers aren't cheap, especially interconnecting them with others.
- m101 2y agoIsn't Moore's law about flops/watt nowadays? Yes that skips the Nvidia part which is $/(flops/watt) but they should just price it at a level where they maximise profits (which is also a function of competitive environment, so not relevant for Moore's law). The reason it's about flops/watt these days is that potentially the demand for "intelligence" limitless.
- SirMaster 2y agoSurely there will be a sizeable increase in performance from the die shrink from 4NP to 2NP.