5 ms·
"For over a decade prophets have voiced the contention that the organization of a single computer has reached its limits and that truly significant advances can
by ZephyrP 15y ago
"For over a decade prophets have voiced the contention that the organization of a single computer has reached its limits and that truly significant advances can be made only by interconnection of a multiplicity of computers in such a manner as to permit cooperative solution."
This now pithy statement was written by the famous Gene Amdahl in the year 1968; A time when computers ran at speeds that are dwarfed by today's digital clocks, but also it gives us insight into a time when people were still dealing with the same problems that we deal with today in developing faster and faster CPUs.
The truth of statement may be something that functionalism or parallelism advocates don't want to hear - the so called Parallelism Revolution will never come, at least not in it's current incarnation.
The end of serial advancement, and thus the parallelization revolution was "supposed" to happen in the 80s, and despite the considerable advances in methodologies of parallelization, it did not come. The 90s brought us standards and technologies like MPI which standardized procedures in developing cooperative computing solutions, but still, it did not come. The 2000s sought to simplify the very act of programming by reappropriating the ideas of programming back to the realm of pure mathematics - by representing programs as a mathematical description of time and work itself, with languages like Haskell and ML we sought to build machines which model math, and thus, the parallel nature of computation within universe itself.
I feel it myself, the sublime glitter of gold that is locked in the idea of parallel computation - It is irresistible for a curious individual. To feel as if all the power of the world is in your hands in this moment (as opposed to 20 years from now), to wipe away the frailty that underlies all of computation today; We all would like to be able to lift a trillion billion bytes into the heavens.
Theres only two problems.
The first problem lies squarely within our own human inadequacies, and it could be argued that this is where parallelism fails deepest. It is certainly true that parallelization is complex, but like all things, abstractions of complexity are nessasary, and designing the abstractions in such a way they are understandable to 'mere mortals' is a greatly undervalued aspect of technology today. So, I would posit as a result of insufficient desire to establish simplified abstractions of parallelization, to most programmers, ideas like parallelism remains in the domain of machine learning and condensed solids analysis - A kind of electronic black art, only used by those with sufficient training to know what horrors they may wrought upon the world if they're to make some trivial programming mistake. As a result (ceteris paribus!) serial power will always be valued greater than parallel computational capacity, which many have claimed to be the predominant driver of commercial development of scientific ideas.
The second problem is more controversial, but I think time will prove it so -- Computer have managed and will continue to manage getting faster at an alarming rate. Regardless of our preconceptions about the mechanics of computation, I believe it is reasonable to say that computers will continue to get faster at exponential rates, even after the so called quantum limits of computation come into play. This is reasonable for the same reason the Normal distribution manifests itself in disparate natural phenomenon - Central Limit Theorem. Sutter himself admits that people have been using the exact same logic to claim the beginning of the end for the past 60-70 years (Before 'real' computers even), I fail to see where he justifies his reasoning after giving this enlightened point.
- rue 15y ago> The truth of statement may be something that functionalism or parallelism advocates don't want to hear - the so called Parallelism Revolution will never come, at least not in it's current incarnation. Do you want to bet everything on a possible advancement of technology, rather than invest in developing better parallelization paradigms? Even now, the state of the art is far beyond manually managing locks and so on.
- jerf 15y agoYour argument would be a lot more compelling if computers were, you know, getting faster. The idea that they might someday stop getting faster is out of date, in the sense that they stopped getting faster at least five years ago, and that's being very conservative. That people decades in the past were wrong doesn't do anything about the fact the people one decade in the past were right. Computers have already stopped getting faster at an "alarming rate", it's a past event, it's not speculation. They're still improving and there's still some room for improvement, but we've already fallen off the exponential curve and I don't anticipate getting back on it anytime soon.
- sounds 15y agoA good point, but all it has done is upped the stakes. Who knows whether single-threaded performance has reached a real barrier or not? Will it be like the sound barrier? Or more like the speed of light? (I realize that the problem is, literally, the speed of light, among other quantum effects.)
- jquery 15y agoI'm not sure what you're talking about, because even for single-threaded applications, my current computer is a couple orders of magnitude faster than the computer I had 5 years ago.
- Steko 15y agoHe's talking simply about clock speed which used to make regular gains with Moore's law but ran up against the heat limits of silicon awhile back. The original article that this follows on gives additional background: http://www.gotw.ca/publications/concurrency-ddj.htm http://www.gotw.ca/publications/concurrency-ddj.htm