3 ms·

We've been doing concurrency for many years now; it's called pthreads. Large applications like Linux, web browsers, webservers, and databases do it all the time

We've been doing concurrency for many years now; it's called pthreads. Large applications like Linux, web browsers, webservers, and databases do it all the time. The question is: how do we design a runtime that makes it harder for the user to introduces races without sacrificing performance or control? One extreme approach is to constrain the user to write only purely functional code, and auto-parallelize everything, like Haskell does (it's obvious why this is a theoretical exercise). Another is to get rid of all shared memory and restrict all interaction between threads to message passing like Erlang does (obviously, you have to throw performance out the window). Yet another approach is to run independent threads and keep polling for changes at a superficial level (like Node.js does; performance and maintainability is shot). The approach that modern languages are taking is to build concurrency as a language primitive built into the runtime (see how go's proc.c schedules various channels in chan.c; it has a nice race detection algorithm in race.c). There is more pressure than ever to build applications that leverages more cores to build highly available internet applications. Multi-cores have existed long enough, and are now prevalent even on mobile devices. No radically different solution to concurrency is magically going to appear tomorrow: programmers _need_ to understand concurrency, and work with existing systems.

seanmcdirmid 13y ago
There are other options in the systems field like "virtual time" and "time warps", or "space-time memory", or a plethora of optimistic concurrency schemes where you optimistically try to do something, discover there is an inconsistency, rollback your effects, and do it again (like STM, but with real "do it again"). Our raw parallel concurrency tools, especially pthreads and..gack..locks, are horribly error prone and not even very scalable in terms of human effort and resource utilization. That is why we've expended so much effort designing models that try and avoid them.
- artagnon 13y ago
  My point is that we'll continually find better solutions to existing problems (concurrency, or anything else for that matter). There will be a time in the future when we've come up with a solution that's "good enough", and it'll become the de-facto standard for a while (kind of like what Java is today). I don't know what that solution will be, and I don't speculate about it: I'm more interested in the solutions we have today. Yes, the raw solutions _are_ very painful, which is why they haven't seen widespread adoption. And yes, we are continually trying to enable more programmers.
  - seanmcdirmid 13y ago
    Yes, many of us are in the field of coming up with "the programming model" to handle this as well as general live programming problems. I'm personally focusing on optimistic techniques to deal with concurrency as well as incremental code changes.
slacka 13y ago
> We've been doing concurrency for many years now; it's called pthreads. Sometimes, the major advances come when fresh ideas are infused from the outside. In Darwin's case it was his geological work that inspired his theory. In concurrency maybe it will be ideas from neuroscience. > No radically different solution to concurrency is magically going to appear tomorrow: programmers _need_ to understand concurrency, and work with existing systems. The environment is changing. In 2007, the oxygen levels started increasing, single threaded CPU scaling hit the wall. It has gone from doubling every 2 years to a few % increases per year. We are only at the beginning of this paradigm shift to massively multi-core CPUs. Both the tools and the theory are still in their infancy. In HW there are many promising advances being explored, such as GPUs, Intel Phi, new FPGAs, and projects like Parallella. The software side also requires new tools to drive these new technologies. Maybe a radical new idea, but more likely some evolved form of CSP, functional, flow-Based, and/or reactive programming models from the 70s, that didn't work with the HW environment at the time will fill this new niche. For example, one of the smartest guys I know working on a neuromorphic engineering where he's creating a ASIC with thousand of cores now and may evolve to (b)millions. If this trilobite emerges on top, whatever language is used to program it might have been terrible in the 70s or for your "existing systems" but it may be the future of programming.
- artagnon 13y ago
  > Sometimes, the major advances come when fresh ideas are infused from the outside. I agree with this largely; over-specialization leads to myopia (often accompanied by emotional attachment to one's work). > In Darwin's case it was his geological work that inspired his theory. If you read On the Origin of Species, you'll see that Darwin started from very simple observations about cross-pollination leading to hybrid plant strains. He spent years studying various species of animals. In the book, he begins out very modestly, following step by step from his Christian foundations, without making any outrageous claims. The fossils he collected on his Beagle expedition sparked his interest in the field, and served as good evidence for his theory. > In concurrency maybe it will be ideas from neuroscience. Unlikely, considering what little we know about the neocortex. The brain is not primarily a computation machine at all; it's a hierarchical memory system that makes mild extrapolations. There is some interest in applying what we know to computer science, but I've not seen anything concrete so far (read: code; not some abstract papers). > We are only at the beginning of this paradigm shift to massively multi-core CPUs. From the point of view of manufacturing, it makes most sense. It's probably too expensive to design and manufacture a single core in which all the transistors dance to a very high clock frequency. Not to mention power consumption, heat dissipation, and failures. In a multi-core, you have the flexibility to switch off a few cores to save power, run them at different clock speeds, and cope with failures. Even from the point of view of Linux, scheduling tons of routines on one core can get very complicated. > In HW there are many promising advances being explored, such as GPUs, Intel Phi, new FPGAs, and projects like Parallella. Ofcourse, but I don't speculate much about the distant future. The fact of the matter is that silicon-based x86 CPUs will rule commodity hardware in the foreseeable future. > [...] All this speculation is fine. Nothing is going to happen overnight; in the best case, we'll see an announcement about a new concurrent language on HN tomorrow, which might turn into a real language with users after 10 years of work ;) I'll probably participate and write patches for it. For the record, Go (which is considered "new") is over 5 years old now.

seanmcdirmid 13y ago

There are other options in the systems field like "virtual time" and "time warps", or "space-time memory", or a plethora of optimistic concurrency schemes where you optimistically try to do something, discover there is an inconsistency, rollback your effects, and do it again (like STM, but with real "do it again"). Our raw parallel concurrency tools, especially pthreads and..gack..locks, are horribly error prone and not even very scalable in terms of human effort and resource utilization. That is why we've expended so much effort designing models that try and avoid them.

artagnon 13y ago
My point is that we'll continually find better solutions to existing problems (concurrency, or anything else for that matter). There will be a time in the future when we've come up with a solution that's "good enough", and it'll become the de-facto standard for a while (kind of like what Java is today). I don't know what that solution will be, and I don't speculate about it: I'm more interested in the solutions we have today. Yes, the raw solutions _are_ very painful, which is why they haven't seen widespread adoption. And yes, we are continually trying to enable more programmers.
- seanmcdirmid 13y ago
  Yes, many of us are in the field of coming up with "the programming model" to handle this as well as general live programming problems. I'm personally focusing on optimistic techniques to deal with concurrency as well as incremental code changes.

slacka 13y ago

> We've been doing concurrency for many years now; it's called pthreads. Sometimes, the major advances come when fresh ideas are infused from the outside. In Darwin's case it was his geological work that inspired his theory. In concurrency maybe it will be ideas from neuroscience. > No radically different solution to concurrency is magically going to appear tomorrow: programmers _need_ to understand concurrency, and work with existing systems. The environment is changing. In 2007, the oxygen levels started increasing, single threaded CPU scaling hit the wall. It has gone from doubling every 2 years to a few % increases per year. We are only at the beginning of this paradigm shift to massively multi-core CPUs. Both the tools and the theory are still in their infancy. In HW there are many promising advances being explored, such as GPUs, Intel Phi, new FPGAs, and projects like Parallella. The software side also requires new tools to drive these new technologies. Maybe a radical new idea, but more likely some evolved form of CSP, functional, flow-Based, and/or reactive programming models from the 70s, that didn't work with the HW environment at the time will fill this new niche. For example, one of the smartest guys I know working on a neuromorphic engineering where he's creating a ASIC with thousand of cores now and may evolve to (b)millions. If this trilobite emerges on top, whatever language is used to program it might have been terrible in the 70s or for your "existing systems" but it may be the future of programming.

artagnon 13y ago
> Sometimes, the major advances come when fresh ideas are infused from the outside. I agree with this largely; over-specialization leads to myopia (often accompanied by emotional attachment to one's work). > In Darwin's case it was his geological work that inspired his theory. If you read On the Origin of Species, you'll see that Darwin started from very simple observations about cross-pollination leading to hybrid plant strains. He spent years studying various species of animals. In the book, he begins out very modestly, following step by step from his Christian foundations, without making any outrageous claims. The fossils he collected on his Beagle expedition sparked his interest in the field, and served as good evidence for his theory. > In concurrency maybe it will be ideas from neuroscience. Unlikely, considering what little we know about the neocortex. The brain is not primarily a computation machine at all; it's a hierarchical memory system that makes mild extrapolations. There is some interest in applying what we know to computer science, but I've not seen anything concrete so far (read: code; not some abstract papers). > We are only at the beginning of this paradigm shift to massively multi-core CPUs. From the point of view of manufacturing, it makes most sense. It's probably too expensive to design and manufacture a single core in which all the transistors dance to a very high clock frequency. Not to mention power consumption, heat dissipation, and failures. In a multi-core, you have the flexibility to switch off a few cores to save power, run them at different clock speeds, and cope with failures. Even from the point of view of Linux, scheduling tons of routines on one core can get very complicated. > In HW there are many promising advances being explored, such as GPUs, Intel Phi, new FPGAs, and projects like Parallella. Ofcourse, but I don't speculate much about the distant future. The fact of the matter is that silicon-based x86 CPUs will rule commodity hardware in the foreseeable future. > [...] All this speculation is fine. Nothing is going to happen overnight; in the best case, we'll see an announcement about a new concurrent language on HN tomorrow, which might turn into a real language with users after 10 years of work ;) I'll probably participate and write patches for it. For the record, Go (which is considered "new") is over 5 years old now.