4 ms·

I find it interesting that the language that best deals with parallelism (IMO) was invented significantly before we moved in the multi-core direction. The prog

I find it interesting that the language that best deals with parallelism (IMO) was invented significantly before we moved in the multi-core direction. The programming language landscape evolved and developed in the face of multi-code - async being a classic example. But the language that's often held up as the best solution to any given parallelism problem is Erlang. Erlang was built as a good programming model for concurrency on a single core, and then when multi-core came along, SMP was 'just' a VM enhancement and no programs needed changing at all to get the full advantage of all the cores (barring some special cases).

deleted 5y ago
[deleted]
ashleyn 5y ago
The writing was on the wall as early as the beginning of the Pentium 4 era. There was this assumption that clock speed would just continue to grow at the insane rates it did in the 80s and 90s, and then reality caught up when the Netburst architecture didn't scale well enough to avoid heat issues and crazy insane pipelines. When Intel rolled out the Core architecture, they went back to a P6-derived design, a microarchitecture that dated back to 1995 - effectively an admission that Netburst had failed.
- lostcolony 5y ago
  Erlang was begun in 1986. It left the lab and was used for production products first in 1996. The Pentium 4 came out in 2000. So, an interesting aside, but not sure it's relevant to the parent comment.
- dleslie 5y ago
  And initial forays into many cores was decades earlier then that. Aside from big iron, going back to the early 60s, there was some business class devices like those made by Sequent: https://en.m.wikipedia.org/wiki/Sequent_Computer_Systems https://en.m.wikipedia.org/wiki/Sequent_Computer_Systems
chrisseaton 5y ago
> no programs needed changing at all to get the full advantage of all the cores (barring some special cases) Some programs just don't have any available parallelism in them - no matter how many processes you split them up into. You need to build parallel algorithms and data structures. That's still often a research-level topic. Erlang's not going to magic parallelism out of thin air in a program that doesn't have any.
- lostcolony 5y ago
  Kind of strawmanning here, no? Obviously sequential tasks are run sequentially. The OP isn't saying "magic" happens. Just, Erlang was written from the beginning to encourage parallelized code. Its whole programming model is around that. Unlike pretty much every language it was contemporary with, which assumed that the default was a single unit of execution (and that code running in parallel was the exception). Think of task scheduling; the normal Algol/C family of languages would say "create a priority queue for your tasks, wait until the top of the queue is ready to run, pull it off, run it, repeat". Very sequential thinking. Of course, you then end up with issues around what happens if the processes take non-negligible amounts of time, what happens if you need to add new items into the queue, etc. At the lowest level that might be solved with interrupts, at a higher level it might be threads or separate processes, but you still have to worry about concurrent data access (locks and the like around the queue). But both the general approach, and the 'best practice' guidelines, especially prior to multi-core, (due to the difficulties of shared data), were/are to minimize concurrency. Erlang, you just spin up every task as its own process. The process sleeps until it's time to do something, then it does it. The language encourages you to solve the problem in a parallel manner, and it did so from the beginning. So, yes, you "need to build parallel algorithms and data structures". The point is, Erlang took advantage of them, as a language, even before the hardware could. The way you would write Erlang for a single core was the way you would write Erlang for many cores, which was unique for a language at the time, and the OP is arguing that not only was Erlang unique in doing that, but that there really isn't a better model for handling parallelism from a language and correctness standpoint (presumably; from a performance perspective m:n threading, of which Erlang actors are a form, is generally agreed upon to lose some efficiency).
kitd 5y ago
But the language that's often held up as the best solution to any given parallelism problem is Erlang. Occam implements a lot of similar ideas very neatly too. The code looks a lot like programming with Go channels and CSP. I am surprised it hasn't caught on better in the multi-core world, it being older than even Erlang.
- acchow 5y ago
  > I am surprised it hasn't caught on better in the multi-core world, it being older than even Erlang. Because multicore support in Ocaml has been "just a couple years away" for many years now.
  - krylon 5y ago
    Occam, not Ocaml. They are two different languages. Although I have no idea if Occam can utilize multiple cores.
    - TickleSteve 5y ago
      Occam was designed for the Transputer, which was intended to be used in arrays of processors, hence it being based on CSP principles. So, yes. it can.
      - krylon 5y ago
        TIL Thank you!
    - acchow 5y ago
      Oops misread that.
- Jtsummers 5y ago
  I had to double check, the last Occam release was in 1994. I think the language was way to niche, and global communication way too poor still (compared to the '00s and '10s with ubiquitous or near ubiquitous internet access) to catch on. It was nearly a decade later that the first consumer multicore x86 processors started coming out (2003 if I've tracked down the right dates). And multicore didn't become common for a few more years and then the baseline near the end of the '00s, beginning of the '10s. That time gap and relative unknown of Occam pretty much doomed it even if it would have been wildly useful. By that point you had lots of other languages to come out, and Google pushing Go.

deleted 5y ago

[deleted]

ashleyn 5y ago

The writing was on the wall as early as the beginning of the Pentium 4 era. There was this assumption that clock speed would just continue to grow at the insane rates it did in the 80s and 90s, and then reality caught up when the Netburst architecture didn't scale well enough to avoid heat issues and crazy insane pipelines. When Intel rolled out the Core architecture, they went back to a P6-derived design, a microarchitecture that dated back to 1995 - effectively an admission that Netburst had failed.

lostcolony 5y ago
Erlang was begun in 1986. It left the lab and was used for production products first in 1996. The Pentium 4 came out in 2000. So, an interesting aside, but not sure it's relevant to the parent comment.
dleslie 5y ago
And initial forays into many cores was decades earlier then that. Aside from big iron, going back to the early 60s, there was some business class devices like those made by Sequent: https://en.m.wikipedia.org/wiki/Sequent_Computer_Systems https://en.m.wikipedia.org/wiki/Sequent_Computer_Systems

chrisseaton 5y ago

> no programs needed changing at all to get the full advantage of all the cores (barring some special cases) Some programs just don't have any available parallelism in them - no matter how many processes you split them up into. You need to build parallel algorithms and data structures. That's still often a research-level topic. Erlang's not going to magic parallelism out of thin air in a program that doesn't have any.

lostcolony 5y ago
Kind of strawmanning here, no? Obviously sequential tasks are run sequentially. The OP isn't saying "magic" happens. Just, Erlang was written from the beginning to encourage parallelized code. Its whole programming model is around that. Unlike pretty much every language it was contemporary with, which assumed that the default was a single unit of execution (and that code running in parallel was the exception). Think of task scheduling; the normal Algol/C family of languages would say "create a priority queue for your tasks, wait until the top of the queue is ready to run, pull it off, run it, repeat". Very sequential thinking. Of course, you then end up with issues around what happens if the processes take non-negligible amounts of time, what happens if you need to add new items into the queue, etc. At the lowest level that might be solved with interrupts, at a higher level it might be threads or separate processes, but you still have to worry about concurrent data access (locks and the like around the queue). But both the general approach, and the 'best practice' guidelines, especially prior to multi-core, (due to the difficulties of shared data), were/are to minimize concurrency. Erlang, you just spin up every task as its own process. The process sleeps until it's time to do something, then it does it. The language encourages you to solve the problem in a parallel manner, and it did so from the beginning. So, yes, you "need to build parallel algorithms and data structures". The point is, Erlang took advantage of them, as a language, even before the hardware could. The way you would write Erlang for a single core was the way you would write Erlang for many cores, which was unique for a language at the time, and the OP is arguing that not only was Erlang unique in doing that, but that there really isn't a better model for handling parallelism from a language and correctness standpoint (presumably; from a performance perspective m:n threading, of which Erlang actors are a form, is generally agreed upon to lose some efficiency).

kitd 5y ago

But the language that's often held up as the best solution to any given parallelism problem is Erlang. Occam implements a lot of similar ideas very neatly too. The code looks a lot like programming with Go channels and CSP. I am surprised it hasn't caught on better in the multi-core world, it being older than even Erlang.

acchow 5y ago
> I am surprised it hasn't caught on better in the multi-core world, it being older than even Erlang. Because multicore support in Ocaml has been "just a couple years away" for many years now.
- krylon 5y ago
  Occam, not Ocaml. They are two different languages. Although I have no idea if Occam can utilize multiple cores.
  - TickleSteve 5y ago
    Occam was designed for the Transputer, which was intended to be used in arrays of processors, hence it being based on CSP principles. So, yes. it can.
    - krylon 5y ago
      TIL Thank you!
  - acchow 5y ago
    Oops misread that.
Jtsummers 5y ago
I had to double check, the last Occam release was in 1994. I think the language was way to niche, and global communication way too poor still (compared to the '00s and '10s with ubiquitous or near ubiquitous internet access) to catch on. It was nearly a decade later that the first consumer multicore x86 processors started coming out (2003 if I've tracked down the right dates). And multicore didn't become common for a few more years and then the baseline near the end of the '00s, beginning of the '10s. That time gap and relative unknown of Occam pretty much doomed it even if it would have been wildly useful. By that point you had lots of other languages to come out, and Google pushing Go.