3 ms·
"For over a decade prophets have voiced the contention that the organization of a single computer has reached its limits and that truly significant advances can
by ZephyrP 15y ago
"For over a decade prophets have voiced the contention that the organization of a single computer has reached its limits and that truly significant advances can be made only by interconnection of a multiplicity of computers in such a manner as to permit cooperative solution."
This now pithy statement was written by the famous Gene Amdahl in the year 1968; A time when computers ran at speeds that are dwarfed by today's digital clocks, but also it gives us insight into a time when people were still dealing with the same problems that we deal with today in developing faster and faster CPUs.
The truth of statement may be something that functionalism or parallelism advocates don't want to hear - the so called Parallelism Revolution will never come, at least not in it's current incarnation.
The end of serial advancement, and thus the parallelization revolution was "supposed" to happen in the 60s, and despite the considerable advances in methodologies of parallelization, it did not come. The 90s brought us standards and technologies like MPI which standardized procedures in developing cooperative computing solutions, but still, it did not come. The 2000s sought to simplify the very act of programming by reappropriating the ideas of programming back to the realm of pure mathematics - by representing programs as a mathematical description of time and work itself, with languages like Haskell and ML we sought to build machines which model math, and thus, the parallel nature of computation within universe itself.
I feel the curious allure of parallel computing myself, a sublime glitter of gold that is locked in the idea - It is irresistible for a curious individual. To feel as if all the power of the world is in your hands in this moment (as opposed to 20 years from now), to wipe away the frailty that underlies all of computation today; We all would like to be able to lift a trillion billion bytes into the heavens.
Theres only two problems.
The first problem lies squarely within our own human inadequacies, and it could be argued that this is where parallelism fails deepest. It is certainly true that parallelization is complex, but like all things, abstractions of complexity are nessasary, and designing the abstractions in such a way they are understandable to 'mere mortals' is a greatly undervalued aspect of technology today. So, I would posit as a result of insufficient desire to establish simplified abstractions of parallelization, to most programmers, ideas like parallelism remains in the domain of machine learning and condensed solids analysis - A kind of electronic black art, only used by those with sufficient training to know what horrors they may wrought upon the world if they're to make some trivial programming mistake. As a result (ceteris paribus!) serial power will always be valued greater than parallel computational capacity, which many have claimed to be the predominant driver of commercial development of scientific ideas.
The second problem is more controversial and is much more material, but I think time will prove it so -- Computer have managed and will continue to manage getting faster at an alarming rate. Regardless of our preconceptions about the mechanics of computation, I believe it is reasonable to say that computers will continue to get faster at exponential rates, even after the so called quantum limits of computation come into play. This is reasonable for the same reason the Normal distribution manifests itself in disparate natural phenomenon - The Central Limit Theorem. Anyone writing about 'CPU slowdown' admits that people have been saying the same thing for 70 years (Even before 'real' computers), and I fail to see where they justify their reasoning that somehow this century is different.