15 ms·
The End of Moore’s Law
- TrickyRick 9y agoDoes anyone have a summary?
- ci5er 9y agoLearn to read? EDIT: I try to not degrinitgrate the illiterate, but as the first commenter on this thread (you), I've gotta go "gosh!" Please tell me how I am wrong. (It's possible that I am both wrong and having a bad day)
- no_gravity 9y agoThe author argues that the number of meaningful operations a device can do per second will not keep growing exponentially. The reason he gives is that while we can add more parts that work in parallel, this has limited benefit: The speed up starts to disappear as silicon is left idle because there just aren’t enough different things to do. I think there will be enough 'things to do'. Looking at real world applications of more processing power like AI and simulations, I expect them to be perfectly parallelizable.
- ianhowson 9y ago> I expect them to be perfectly parallelizable Unfortunately, the real world does not meet your expectation. You're talking about a class of problems called 'embarrassingly parallel': https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel Only a very tiny proportion of problems fit into that category. AI and simulations tend to be more easily parallelizable, but CPU-CPU communication is slow at any scale and imposes a limit to how much parallelism can be exploited. The original statement is accurate: > The speed up starts to disappear as silicon is left idle because there just aren’t enough different things to do Efficient parallel algorithms require a problem that can be divided into smaller independent components and solved separately. Not every problem can be divided in this way. Crypto algorithms are the classic example; there's no way to perform round N+1 without first performing round N, and this is very much by design.
- threatofrain 9y agoI wonder if those stubbornly serial problems may be arbitrarily approximated by parallel solutions.
- hyperpallium 9y agoOr a less efficient (per computation) parallel version, that scales.
- ianhowson 9y agoA parallel algorithm is always less efficient (per unit of computational resources) due to the extra communication overhead. That's the tradeoff for being able to utilise more total computational resources.
- hyperpallium 9y agoSure; I meant a different algorithm. e.g. inefficient by recomputing the same result in some cases; perhaps like parallel searches that may overlap. Such strategies would not have been researched much because they seem not just inefficient but stupid. But when you have cores to burn, they may turn out to be clever... I know of a case where similar happened when silicon became much cheaper several decades ago... the inventor said it just wouldn't have occurred to anyone before then, because it required an infeasible amount of silicon. People couldn't (or wouldn't) think in those terms.
- eleitl 9y ago> Only a very tiny proportions fit into that category Actually, it is exactly the other way round: most problems in the real world are about local communications of cells in a 3 dimensional lattice. Including, relativistic limits to communication. So an ideal architecture for that is a 3D cellular automaton. With bigger cells, you've got nodes on a 3d mesh (torus). Incidentally, the topology of most supercomputers.
- 9y ago
- tzahola 9y agoAmdahl’s law
- zaarn 9y agoSadly, in the real world most silicon for users will be busy rendering a cat trying to fit into a box and then falling over or alternatively attempting and failing to properly layout a word document while also accommodating macros inside said word document. Word documents and tumbling cats don't really benefit from more than a few cores.
- librvf 9y ago> Sadly, in the real world most silicon for users will be busy rendering a cat trying to fit into a box and then falling over Better than ideological proselytizing and political propaganda.
- imtringued 9y agoOn the other hand they don't need more than one good core.
- doomlaser 9y agohttps://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law
- grogers 9y agohttps://en.m.wikipedia.org/wiki/Gustafson%27s_law https://en.m.wikipedia.org/wiki/Gustafson%27s_law
- zwischenzug 9y agoI recently wrote this article on a book from 1965 called 'Electronic Computers' https://zwischenzugs.com/2017/11/11/towards-a-national-computer-grid-electronic-computers-1965/ https://zwischenzugs.com/2017/11/11/towards-a-national-compu... In there is an image from 1965 of a graph similar to Moore's Law: https://zwischenzugs.files.wordpress.com/2017/11/20171105_1137192-e1510393753138.jpg?w=840 https://zwischenzugs.files.wordpress.com/2017/11/20171105_11... I'd be interested to know if anyone has the first edition of this book, or whether there were other similar graphs floating around at this time. Note that the image also shows a storage graph alongside computing power. Does anyone know if this graph was continued, or extended into a 'law' also?
- BoiledCabbage 9y agoThat's roughly the same rate as Moore's law. 100x every 10 years.
- larkeith 9y agoOh goody, it's time for the annual 'End of Moore's Law' article. http://www.sciencedirect.com/science/article/pii/S0375960102013658 http://www.sciencedirect.com/science/article/pii/S0375960102...
- tzahola 9y agoReminds me of IPv4 address space exhaustion articles of the past.
- PakG1 9y agoThat's gotta finally happen SOME day, no?? I speak seriously.... Of course, by the time it happens, I wouldn't be surprised if we come up with a solution that people like better than IPv6?
- clarry 9y agoIt happened already. And nothing changed.
- ghaff 9y ago>And nothing changed. It sort of has. We've shifted a lot of computing to the back-end where we can use more servers. We're using alternative architectures that can be built into chips that are more efficient for some workloads, e.g. ARM. We're using GPUs and now TPUs for compute-intensive tasks like machine learning that can benefit from them. It's not clear to what degree we'll be able to replace process scaling with other approaches but certainly things will change with chips unable to just get denser/faster at a predictable rate any longer.
- clarry 9y agoI fail to see the connection to the exhaustion of the IPv4 address space.
- notheguyouthink 9y ago
- megaman22 9y agoI dunno, it's very different than things were. I built my current desktop in 2011, and aside from an ssd and upgrading from a 560 to a 970 gtaphics card, haven't made any changes. I'm running everything I want, multiple VMs all the time, Visual Studio, games at max graphics. Ten years ago, using a six year old machine meant you wouldn't have a snowball's chance in hell of running new software, but this one is chugging along admirably, and probably will for the foreseeable future. Unless everything moves to Electron, but if that happens, it really is the end times.
- runeks 9y ago> Ten years ago, using a six year old machine meant you wouldn't have a snowball's chance in hell of running new software [...] Do you have some evidence for this claim? I'm not saying I disagree -- indeed, I suspect it might be true -- but I'd appreciate some evidence to back it up.
- megaman22 9y agoTwo that come to mind are Windows Vista upgrades, where many XP-capable machines were just not powerful enough to handle it. The other would be video games, which were constantly pushing the envelope. Consoles may have a lot to do with the stagnation in the latter case, however.
- Brockenstein 9y agoRight. I bought a Pentium 3 800/GeForce 2 GTS in the summer of 2000. And by 2003 Homeworld 2 and Knights of the Old Republic had a Pentium 3 833mhz and a Pentium 3 1ghz respectively.
- ptero 9y agoI do not have statistically significant evidence, but as another data point I have the same experience: both my home and work machines are 7-8 years old and work just fine (30" monitors, decent performance with numpy, MATLAB, compiling, perfectly fine graphics, etc.). But around 2000 I remember always carefully looking at the system requirements section of games to check which of those would work on my 2-3 year old computer and for which my machine is too old -- and find a significant rejection rate (25-50%).
- juanmirocks 9y agoThe end of Moore's Law doesn't necessarily have to imply a deceleration in technological evolution, as Ray Kurzweil and others have pointed out. In their view, Moore's Law is just one single phase in the "law of accelerating returns".
- ChrisSD 9y agoWhich is what this article also argues. As we can no longer rely on Moore's exponential increases we will start looking elsewhere to evolve technology. A new "golden age" as he calls it. Maybe that'll mean looking beyond von neumann for general purpose architecture. Maybe it'll mean designing a lot more specialised architectures, similar to how the GPU is specialised for graphics processing (and useful for some other tasks) or how most mobile devices come with specialised processors. Maybe it'll mean something completely different. The future of computing is exciting again.
- hyperpallium 9y agoLike peak oil. But there might be a (seemingly) interminable lull. Meanwhile, I like the idea of physically massive GPUs if they can't get smaller (also doing ML, CFD etc)
- bluddy 9y agoThat makes little sense. We've been riding this particular technology called transistors we've stumbled on for the past 60 or so years. Now the technology is mature, and to make serious progress, we have to get lucky again and stumble on another technology. It doesn't just happen on its own.
- amelius 9y agoIf only Moore's law applied to clock speed instead of some useless measure like number of transistors.
- runeks 9y agoThen what?
- deleted 9y ago[deleted]
- singularity2001 9y agoIf only Moore's law applied to 'calculations per second', then every gamer or AI enthusiast could tell stories about the recent progress of GPUs.
- amelius 9y agoIt's easy to increase number of calculations per second. I.e., simply increase the number of transistors. It's not so easy to increase clock speed.
- qznc 9y agoThat is more like Dennard's law [0] but that one stopped roughly 2001. You could also watch Koomey's law [1]. [0] https://en.wikipedia.org/wiki/Dennard_scaling https://en.wikipedia.org/wiki/Dennard_scaling [1] https://en.wikipedia.org/wiki/Koomey%27s_law https://en.wikipedia.org/wiki/Koomey%27s_law
- Clubber 9y agoMoore's law is number of transistors. Moore's law is the observation that the number of transistors in a dense integrated circuit doubles approximately every two years. https://en.wikipedia.org/wiki/Moore%27s_law https://en.wikipedia.org/wiki/Moore%27s_law
- tjoff 9y ago
- qznc 9y agoHere is a graph from Wikipedia data until the AMD 8-core Ryzen: https://twitter.com/azwinkau/status/869461530324107264 https://twitter.com/azwinkau/status/869461530324107264 As soon as a plot like this shows a "stop", there will pop up lots of possibilities for research and funding thereof. Physical restrictions predict this stop within the next years.
- eleitl 9y agoExcept that Intel's own data show that the constant doubling no longer applies. I have no idea what he means by Wikipedia data. See no source there.
- wott 9y ago> I have no idea what he means by Wikipedia data. I imagine it is this: https://en.wikipedia.org/wiki/Transistor_count https://en.wikipedia.org/wiki/Transistor_count
- thechao 9y ago> Once you get down to 5nm features they are only about 20 silicon atoms wide. This is a complete misunderstanding of modern processes. It’s conflating feature size with some measure like “wire width”. Feature size is best thought of as the tightest corner radius you can cut in a piece of wood. Actual logic (and wires) are far larger than this. There’s actually a shocking amount of room “down there”, still. The problem is that lithography probably won’t scale down after some magic number (say 2 or 3 nm), and we’ll have to do something else clever. It’s been a while since I looked into this, but IBM’s smallest AFM-created transistor is ~10 atoms, which is something like 4 orders of magnitude (or more!) smaller than, say, 10nm.
- eleitl 9y agoMoreover, the node numbers do no longer relate to any specific geometric feature, but are computed from how many standard (memory) cells fit a given area.
- notheguyouthink 9y agoReally ignorant question, but does size reduction affect heat dissipation at all?
- pjc50 9y agoSort of: thinner wires have higher resistance, so what tends to happen is you get slightly more heat but in a much smaller space. Heat dissipation has become a major limiting factor.
- tlb 9y agoYes. Most of the power goes into charging and discharging the capacitance inherent in each wire and transistor gate every time a signal changes. When you shrink by 2x, the capacitance goes down by about 4x (because it's proportional to area) and power goes down 4x also. (It's not exactly square-law, because the sides of wires have capacitance too, but at least at larger sizes it was close.) See https://en.wikipedia.org/wiki/Dennard_scaling https://en.wikipedia.org/wiki/Dennard_scaling for details.
- 9y ago
- waivek 9y agoOne point that I wish the article had expanded on is the impact on software development. In a recent speech, John Carmack talks of how the power of the desktop will never reach mobile. https://m.youtube.com/watch?v=vlYL16-NaOw&t=14m20s https://m.youtube.com/watch?v=vlYL16-NaOw&t=14m20s This is incredibly exciting. I see the end of Moores law as the return of the era of performance-oriented software development. The past decade was ruled by software methodologies which optimised for developer time at the cost of performance. Moores law was the crutch on which such latency ridden software limped into userland. With the stabilization of hardware performance, developers can once again put dedicated time and effort into crafting performant software with the assurance that hardware gains will not render their efforts useless in 2 years.
- BoiledCabbage 9y ago> This is incredibly exciting. I see the end of Moores law as the return of the era of performance-oriented software development. And I see this as the exact opposite. Optimizing for developer time is one of the largest productivity enhancements we have. A true/pure positive feedback loop in technological progress. Optimizing for performance is getting little real gain for large human cost. The end game of optimized for developer efficiency is we can design and implement "anything" we can think of in almost no time flat. The end game for optimizing for performance is it takes significantly longer to develop a program just to run it on the equivalent of 5 years ago hardware. An analogy: it's early in human writing. People have trouble expressing average thoughts. We can either teach society to have a better grasp on writing and expressing complex ideas precisely. Or we can teach people to write really small because we're running out of paper. The positive feedback loop of improved development is enormous.
- waivek 9y agoIf I may, a rebuttal. The software methodology that has evolved over the last two decades has focused on development time. As such, there has not been much focus/research on performance-first methodologies. I see no reason that developer time cannot be optimized in a performance-first methodology, it's just that a developer time oriented methodology has years of fine tuning in it. An example for this is video game development. If you look at some talks given by Mike Acton, you can see that it is possible to ship a game while optimizing for developer time. A triple A game is arguably one of the most intricate and complex projects under the label of "Software Engineering". I see no reason why we can't learn from such sub-fields and apply it to general software development. I also disagree with you saying that "Optimizing for performance is getting little real gain for large human cost". Not optimizing for performance has had led to the creation of unresponsive text editors, slow word processors and a plethora of other latency-ridden software. This in turn affects the user and unfortunately alters their expectations for software. There is no technological reason why Visual Studio 2017 should not run on a 5 year old laptop, yet that is the state we are in.
- perseusprime11 9y agoHow do we think about AI/ML stack moving into GPUs and custom chips? Maybe Moore's law as envisioned is ending but it is probably changing in new ways to help scale the jobs and tasks we have based on the type of task.
- scarface74 9y agoIt may not be the end of Moore's Law, but the increase in resources needed to run modern software has definitely slowed down. In 2017, my mom is using my old 2006 era Core Duo 1.66Ghz Mac Mini running Windows 7 with only 1.5GB of RAM. She mostly uses it for tutoring when she doesn't want her students on her main computer. It still runs Chrome and Office surprisingly well. It has plenty of USB2 ports, Bluetooth and Gig E Ethernet. i cant imagine getting any use out of the computer I used in 1995 - a PowerMac 6100/60 with a PPC 601/60Mhz processor and a 486Dx/2 66Mhz Dos Compatibility Card in 2006. My Plex Server is a 2008 era Core 2 Duo 2.66Ghz Dell Laptop running Windows 10 with 4Gb of RAM.