9 ms·
The answer was "not while computing was getting exponentially better every year". This sort of improvement curve is really difficult to compete with. It's in t
by jerf 10y ago
The answer was "not while computing was getting exponentially better every year". This sort of improvement curve is really difficult to compete with.
It's in the process of being liberated now. GPUs are already deployed in many machines with a fundamentally different architecture. And there's a bubbling foment of alternatives being developed now.
In the end, I don't think it was necessarily a case of us getting stuck on "von Neumann" per se... we would have gotten stuck on anything we settled on in the 1960s. And despite familiarity breeding contempt, I'm not that convinced we could have done that much better in the 1960s. A lot of the alternatives are not universally better, they're better in some limited domain but it's not obvious that they are going to uniformly beat von Neumann for general computing, even accounting for the fact that "general computing" has coevolved for 50 years with von Neumann.
See also https://news.ycombinator.com/item?id=12155561 https://news.ycombinator.com/item?id=12155561 (which may well have been the inspiration for this submission)
- nickbauman 10y agoNow that Moore's Law has effectively ended on a single chip (ended around 2007), FP is front-and-center again, but mostly because of the emphasis on immutability, which is a prerequisite for parallelization. Moore's Law has been replaced with Amdahl's Law.
- adwn 10y ago> Now that Moore's Law has effectively ended on a single chip (ended around 2007), FP is front-and-center again Ah yes, the decades-old claim that FP will make parallel programming much simpler. I wonder when that notion will go the way of "Java will be faster than C because of the JIT compiler" or "Compilers become better and better, so dynamic scripting languages will be quite fast soon." There are two main problems with this sentiment: 1) Immutability does not automatically result in parallelity. If your algorithm is sequential, there's nothing a functional, immutable language can do to help. Oh, and a lot of algorithms are hard or impossible to parallelize efficiently, and even if you can parallelize it in theory, you still have work with limited latency and throughput to other processing elements. 2) Mutable arrays of compact, pointer-less data elements map incredibly well to transistor arrays (SRAM for caches) and transistor/capacitor arrays (DRAM for main memory). > the emphasis on immutability, which is a prerequisite for parallelization This is blatantly false. Basically all parallel algorithms in high-performance-computing make extensive use of mutable memory.
- nickbauman 10y agoWhat I should have said immutability allows for code that is both amenable to parallelization and be easier to reason about by the people that write it.
- p1esk 10y agoSo if Moore's law is dead since 2007, why do I still use C for my CUDA code, and not some FP language?
- pjlegato 10y agoInertia, mostly. All the common tools are built around C, all the existing code is written in C, all the jobs to maintain and extend that code are all in C. Like most "Why does everyone only ever use language X to do Y" questions, considerations of actual technical merit / best technical fit are a very minor factor in the equation.
- p1esk 10y agoSure, I understand the inertia, but can you show me any demo where a FP language outperforms C for parallel tasks? People in the deep learning field care a lot about speed, and if someone demonstrated a factor of x speedup with FP language, they would use it.
- oldmanjay 10y agoYou want proof of a claim that only you have made. That doesn't seem like a terribly fair criticism of the OP.
- p1esk 10y agoWell, this discussion started when someone suggested the speed increases fueled by Moore's law held FP back. This is what I've been questioning. If it's not about speed, then what does it have to do with Moore's law?
- yosefk 10y agoGPUs are much closer to von Neumann machines than to anything people typically deriding the latter have in mind (such as say the Reduceron architecture.) Every commercially significant accelerator, GPUs included, has a RAM, often multi-banked to support vectorized random access, it has an imperative programming model, and it runs (often vector) instructions from program memory operating on registers, data is occasionally exchanged between registers and RAM addresses. $100 says this will remain true for the next 20 years. The thing most far away from a von Neumann machine that is commercially significant is the FPGA and it is nothing like what people deriding the von Neumann machines have in mind. It's imperative, has registers and RAM blocks, and it will cause nausea in the typical functional programming supremacist. (Except those that choose to squander its performance potential on implementing a wasteful machine like the Reduceron because graph reduction is just better than an imperative low-level programming model.) Programmers need to be liberated from misconceptions wrt von Neumann machines.
- DigitalJack 10y agoI have to ask, what makes you say this: "and it will cause nausea in the typical functional programming supremacist" with regard to FPGAs?
- _yosefk 10y agoNot only are FPGAs as far from the ideal of "computation as reduction" expressed in the paper as they could possibly be, but they're the hardest machine to fit into this sort of a paradigm. The typical CPU will run functional code fairly efficiently if enough work is put into the compiler (to some functional programmers this sounds bad because hardware should make writing functional language compilers easy. People who know about hardware make a different conclusion - that hardware is just fine, as Ken Thompson said, "the PDP-11 is a fine Lisp machine, it's not a language special enough to warrant a specialized machine.") The typical accelerator is a harder target to a general-purpose functional programming language (it's a poor target for a general-purpose imperative language to begin with.) The FPGA, with its explicit management of every imaginable resource, is the worst target for a general-purpose functional programming language. In general, the farther away a commercially significant machine is from the classical von Neumann architecture, the more terrible target it is for functional languages, which never prevented FP aficionados from talking about "the end of Moore's law bringing about the long-awaited demise of the von Neumann machine", a "demise" that largely doesn't happen and where it does, it does not mean what they think it means.
- sliverstorm 10y agoIt's in the process of being liberated now. Reading between the lines- it's not really something to celebrate, given it is happening because progress is slowing. I'd rather be chained to a flawed model and continue to enjoy exponential improvement for a long time.
- zachsnow 10y agoHumans in general prefer this. See also petroleum. And in the same way it seems the only thing that will see a change occur is necessity.
- willtim 10y agoA better example would be any modern SMP x86 machine. These use a quickpath bus to connect distributed memory and cpu, basically a cluster. Intel goes to great pain to give the illusion of a von Neumann machine, but it's a leaky abstraction.
- ktRolster 10y ago>but it's a leaky abstraction. Really? The only time I see anything close to a leak is when I look at the speed of various operations (and let's be honest, von Neumann's machines always had bottlenecks, as much as he tried to keep them out)
- willtim 10y agoAs soon as you run more than one thread, you'll have to worry about memory barriers, cache coherency, processor affinity etc. Even with one thread, modern IO and system calls are highly asynchronous. Not the sort of machine that e.g. C or C++ was originally envisioned for.
- ktRolster 10y agoEven with one thread, modern IO and system calls are highly asynchronous. What are you doing, calling select() or something?