8 ms·

The article is very nice, thanks a lot for it. Especially since I hear the word concurrency and parallelism often thrown around without any distinction. Very o

The article is very nice, thanks a lot for it. Especially since I hear the word concurrency and parallelism often thrown around without any distinction. Very off topic, but I have read several times the argument that the rise of functional programming is due to it's easy concurrency (since functions don't have side effects) and that concurrency becomes more and more important due to moores law being dead (i.e. we can't scale the hardware up, we have to add cores to our processors). Could someone with more experience comment on that? Is concurrency really easier in functional languages and is the rising importance of concurrency a valid reason to look into functional programming?

leethargo 7y ago
I don't have more experience, necessarily, but I think the argument builds on the facts that pure functions don't have side-effects, which avoids many of the problems of simultaneous data editing, and also the use of immutable data structures, which can be shared freely (in a read-only manner).
clarry 7y ago
> Is concurrency really easier in functional languages and is the rising importance of concurrency a valid reason to look into functional programming? I think it depends on who's writing the code. A little bit of shared mutable state here and there isn't gonna kill you, but if the program's architecture is poor, that'll blow up and spread everywhere and the next thing you know is you're spending half your time worrying about data races, deadlocks, and a locking nightmare. Shared-nothing architectures fix that; your synchronization will happen over a narrow range of well defined primitives (e.g. message queues), though this doesn't prevent race conditions. Functional programming that encourages pure functions & immutable state is a bit of a straight jacket that keeps you from making a mess that blows up. I think it helps, and I wish $stuffAtWork were written in such a language because I'm tired of wrestling with terrible architecture (that I'm not really allowed to fix, given the time constraints). The other thing that makes a difference is what your problem looks like. Sometimes concurrency is absolutely trivial, like dispatching sections of an array for threads to perform (independent) compute on, and come back with a result. Sometimes it's way more complicated...
- toolslive 7y ago
  Yes. Concurrency is easier in functional languages. Many of them provide a concurrency monad which allows you to write expressions like this: read_from_socket_into_buffer params >>= process_buffer >>= ... Where you roughly read the '>>=' as follows: "the left side might take an arbitrary amount of time to produce a result. While it's off and doing its thing, go and do something else. Once it has produced the result, take it and feed it as a parameter to function on the right." BTW, It's quite easy to implement your own concurrency monad, however: the real work is the wrapping of all the blocking system calls into this framework.
  - zelphirkalt 7y ago
    This sounds a bit like async in some other languages.
    - dahfizz 7y ago
      Lots of functional language featured end up copied into traditional languages (monads, immutability, pattern matching, lambdas, etc). The difference is that these are almost always more ergonomic in a functional language that was designed with these features in mind vs. having them bolted on afterwards
Athas 7y ago
Concurrency is all about simultaneous effects. A language like Haskell is nice enough because it makes effects explicit, so you at least know where things can go wrong, but if you have a bunch of concurrent effectful Haskell threads running at the same time, you have the usual issues with nondeterminism. Functional languages to tend to emphasize higher-level concurrency patterns than old-school shared mutable state, so in practice it works OK. Specifically, concurrent Haskell is good and also quite fast. It doesn't quite compare to Erlang in the concurrency department although the languages are very different in other ways, so there are reasons to prefer Haskell anyway. I think this shows that just having a functional language that emphasizes effect-free programming is not sufficient. If you want a good concurrent language, then you have to design it like that, and not simply rely on the absence of shared mutable state. In practice, people write massively concurrent systems all the time, in the form of dynamic web sites that handle many concurrent requests, and synchronise access to a shared mutable state via a transactional database of some kind. These don't depend on fancy languages, but on a rigid architecture that tries to isolate synchronisation issues in the web server and database components. While this article is about concurrency, I'll also add a note on parallelism: Parallel functional programming has IMO not been shown to work well in practice. While it is trivial to parallelise more or less any pure functional program correctly, it is very hard to actually gain meaningful speedup from it. The single point of parallelism is performance, and most current implementations of functional languages (including all general purpose functional languages I know of) simply have too many bottlenecks or pitfalls to scale for computational work as well in practice as one might expect in theory. For example, concurrent Haskell is fast because most of the actual work tends to be IO and GHC has an excellent parallel IO manager and scheduler, while parallel Haskell is hampered by, among other things, a slow garbage collector.
- hopia 7y ago
  Do you refer to Erlang excelling in distributed concurrency, due to how its built to rely on the actor model? Or Erlang also excels in concurrency on a single machine? Btw, what are some of those other things hampering Haskell's performance with parallel computing? Wouldn't the garbage collector mainly produce pauses instead of really destroying parallel performance? Although a concurrent garbage collector has recently been merged to the nightly GHC.
zelphirkalt 7y ago
What you heard is correct. If you have pure functions and provided, that you have working primitives of concurrency, you can go to any place in you code, where you call a pure function and change the code, so that the function runs in a new concurrency (or parallelism) primitive. For example in a new thread or process. The places where this is useful can be discovered by profiling. Another advantage is, that you can unit test each function on its own, without setting up a whole host of preconditions. That's because of referential transparency.
01100011 7y ago
I'm just an idiot EE who learned to program, but it seems like functional languages without side-effects accomplish their task by basically working on data structures stored on the stack. That basically means each call stack has its own data structure. If you wrote threaded code so that each thread had its own data structure, and worked off a shared set of read-only data structures, you wouldn't need to worry about locking. IME locking isn't really all that hard until you need to squeeze out performance. If you can handle one big dumb lock around everything, it's easy. The finer grained your locking gets, the harder it is to get right(even then, lock hierarchies can help to a great degree).
- bcrosby95 7y ago
  The problem is that if you have one big dumb lock around everything then you have no actual concurrency or parallelism.
  - 01100011 7y ago
    Depends on what the big dumb lock protects. You may still be able to process data you accessed while holding the big dumb lock.
- unlinked_dll 7y ago
  One big dumb lock only protects against data races. There are other classes of concurrency bugs that require smart locking to avoid, and it's always a pain. For example, iterator invalidation. Which is probably the most common thing I've seen when people use dumb locks to make a concurrent program sound. Either your iterators need to be aware of the lock and hold it through their lifetime (which means practically only one iterator can exist, which is undesirable) or your datastructure needs to be aware of outstanding iterators to it and update them (which means you have added logic to simple mutations, which is also undesirable). It's problems like that you look for in non-functional languages that require library authors to say things like "this is/isn't thread safe" in their documentation, while providing those guarantees manually. And as a consumer of the library you need to be aware of these things and look for them, and what an error looks like when someone messes up. I don't know what you're trying to say about stacks/the call stack in functional languages, because they implement their patterns in variety of ways - including locking. It's just about providing particular guarantees in the API, and the benefits of those guarantees are that it prevents entire classes of bugs like iterator invalidation and data races. There are also all sorts of clever data structures you can use that don't require duplication of the underlying data. The easiest is a singly linked list without insertion. No locks required.
Heliosmaster 7y ago
Yes. I can say that doing concurrency in Clojure is particularly easy, check for yourself: https://www.braveclojure.com/core-async/ https://www.braveclojure.com/core-async/
imtringued 7y ago
You are confusing Moores' Law with Dennard scaling. Without Moores' Law you're not adding more transistors and therefore more cores.
jimbokun 7y ago
> Is concurrency really easier in functional languages You can use the same approaches in non-functional languages. You can only use immutable data structures, and pure functions everywhere. Then you get the same benefits of easier reasoning about concurrent execution, as you get from a functional programming language. So the point of using an actual functional language, is that the compiler pushes you very strongly towards programming this way. It's kind of like using a language with built in garbage collection. You could implement a garbage collector in C, and use it everywhere in your program where you allocate and release memory. But having it build into the language frees you up from having to think about it very much.
- dirtydroog 7y ago
  I don't get the immutable data structures thing though. If I have a vector of 1 billion numbers and I need to sort it it'll be unfeasible to make a copy every time I swap elements. RAM isn't infinite either. So, deep down in the guts of the language's libraries there must be mutable operations. Which kind of defeats the whole point. I don't get point of functional programming. Machines don't work that way.
  - tmountain 7y ago
    The implementations of immutable data structures in most concurrency aware functional languages are smarter than that. Take Clojure as an example: "All of the Clojure collections are immutable and persistent. In particular, the Clojure collections support efficient creation of 'modified' versions, by utilizing structural sharing, and make all of their performance bound guarantees for persistent use." https://hypirion.com/musings/understanding-persistent-vector-pt-1 https://hypirion.com/musings/understanding-persistent-vector...
  - moring 7y ago
    IMHO the advantages of functional programming is often looked for in examples that are too small for them to exist. There is no advantage in sorting a billion numbers the functional way. There is an advantage to gain, though, when the unsorted numbers are used for other purposes besides sorting. Then, functional programming demands that the output of sorting is written to a new 1-billion array instead of "in place", so other clients can consume the same array in parallel without being disturbed by the sorting. > So, deep down in the guts of the language's libraries there must be mutable operations. Which kind of defeats the whole point. It doesn't defeat the point if the abstraction doesn't leak. If a pure functional language offers sorting as a pure primitive, then language-wise it doesn't matter that it actually mutates memory internally. The problem with "mostly" pure functional languages is that the abstraction leaks enough to make it useless.
toast0 7y ago
> Could someone with more experience comment on that? Is concurrency really easier in functional languages and is the rising importance of concurrency a valid reason to look into functional programming? Only you know if concurrency is relavant to your work. A lot of internet stuff is deeply concurrent, but there's a lot of sequential work out there, too. I have experience with Erlang. It's amazing for concurrency, but the reason is really not anything it gives you, but everything it prevents you from doing, or requires you to do. Erlang's immutability is important because it means you can't save a reference to something and have it magically updated; if you want a value updated somewhere, you need to send the current value or request the current value at time of use; explicit synchronization makes concurrency clearer, and helps influence system design to reduce synchronization. Immutability also makes message passing easier; the content of a message can't change after it's composed which means the implementation is free to make a copy of the message if it's convenient and as a programmer, there's no need to consider the effects of changing something that you sent to another process. Immutability also makes garbage collection very simple; it's impossible to form a looped datastructure, so there's no need for mark and sweep, a simple copying collector is sufficient. Individual stacks per Erlang process (like a green thread) means GC pauses don't stop the world, only individual processes, and pause length is bounded and proportional to the memory use of the process. You could certainly set up a similar environment in C or whatever traditional language; the problem would be enforcing your system design on your own code, and partitioning off library code (including libc and/or kernel syscalls) to ensure it doesn't impact your system design either.

leethargo 7y ago

I don't have more experience, necessarily, but I think the argument builds on the facts that pure functions don't have side-effects, which avoids many of the problems of simultaneous data editing, and also the use of immutable data structures, which can be shared freely (in a read-only manner).

clarry 7y ago

> Is concurrency really easier in functional languages and is the rising importance of concurrency a valid reason to look into functional programming? I think it depends on who's writing the code. A little bit of shared mutable state here and there isn't gonna kill you, but if the program's architecture is poor, that'll blow up and spread everywhere and the next thing you know is you're spending half your time worrying about data races, deadlocks, and a locking nightmare. Shared-nothing architectures fix that; your synchronization will happen over a narrow range of well defined primitives (e.g. message queues), though this doesn't prevent race conditions. Functional programming that encourages pure functions & immutable state is a bit of a straight jacket that keeps you from making a mess that blows up. I think it helps, and I wish $stuffAtWork were written in such a language because I'm tired of wrestling with terrible architecture (that I'm not really allowed to fix, given the time constraints). The other thing that makes a difference is what your problem looks like. Sometimes concurrency is absolutely trivial, like dispatching sections of an array for threads to perform (independent) compute on, and come back with a result. Sometimes it's way more complicated...

toolslive 7y ago
Yes. Concurrency is easier in functional languages. Many of them provide a concurrency monad which allows you to write expressions like this: read_from_socket_into_buffer params >>= process_buffer >>= ... Where you roughly read the '>>=' as follows: "the left side might take an arbitrary amount of time to produce a result. While it's off and doing its thing, go and do something else. Once it has produced the result, take it and feed it as a parameter to function on the right." BTW, It's quite easy to implement your own concurrency monad, however: the real work is the wrapping of all the blocking system calls into this framework.
- zelphirkalt 7y ago
  This sounds a bit like async in some other languages.
  - dahfizz 7y ago
    Lots of functional language featured end up copied into traditional languages (monads, immutability, pattern matching, lambdas, etc). The difference is that these are almost always more ergonomic in a functional language that was designed with these features in mind vs. having them bolted on afterwards

Athas 7y ago

Concurrency is all about simultaneous effects. A language like Haskell is nice enough because it makes effects explicit, so you at least know where things can go wrong, but if you have a bunch of concurrent effectful Haskell threads running at the same time, you have the usual issues with nondeterminism. Functional languages to tend to emphasize higher-level concurrency patterns than old-school shared mutable state, so in practice it works OK. Specifically, concurrent Haskell is good and also quite fast. It doesn't quite compare to Erlang in the concurrency department although the languages are very different in other ways, so there are reasons to prefer Haskell anyway. I think this shows that just having a functional language that emphasizes effect-free programming is not sufficient. If you want a good concurrent language, then you have to design it like that, and not simply rely on the absence of shared mutable state. In practice, people write massively concurrent systems all the time, in the form of dynamic web sites that handle many concurrent requests, and synchronise access to a shared mutable state via a transactional database of some kind. These don't depend on fancy languages, but on a rigid architecture that tries to isolate synchronisation issues in the web server and database components. While this article is about concurrency, I'll also add a note on parallelism: Parallel functional programming has IMO not been shown to work well in practice. While it is trivial to parallelise more or less any pure functional program correctly, it is very hard to actually gain meaningful speedup from it. The single point of parallelism is performance, and most current implementations of functional languages (including all general purpose functional languages I know of) simply have too many bottlenecks or pitfalls to scale for computational work as well in practice as one might expect in theory. For example, concurrent Haskell is fast because most of the actual work tends to be IO and GHC has an excellent parallel IO manager and scheduler, while parallel Haskell is hampered by, among other things, a slow garbage collector.

hopia 7y ago
Do you refer to Erlang excelling in distributed concurrency, due to how its built to rely on the actor model? Or Erlang also excels in concurrency on a single machine? Btw, what are some of those other things hampering Haskell's performance with parallel computing? Wouldn't the garbage collector mainly produce pauses instead of really destroying parallel performance? Although a concurrent garbage collector has recently been merged to the nightly GHC.

zelphirkalt 7y ago

What you heard is correct. If you have pure functions and provided, that you have working primitives of concurrency, you can go to any place in you code, where you call a pure function and change the code, so that the function runs in a new concurrency (or parallelism) primitive. For example in a new thread or process. The places where this is useful can be discovered by profiling. Another advantage is, that you can unit test each function on its own, without setting up a whole host of preconditions. That's because of referential transparency.

01100011 7y ago

I'm just an idiot EE who learned to program, but it seems like functional languages without side-effects accomplish their task by basically working on data structures stored on the stack. That basically means each call stack has its own data structure. If you wrote threaded code so that each thread had its own data structure, and worked off a shared set of read-only data structures, you wouldn't need to worry about locking. IME locking isn't really all that hard until you need to squeeze out performance. If you can handle one big dumb lock around everything, it's easy. The finer grained your locking gets, the harder it is to get right(even then, lock hierarchies can help to a great degree).

bcrosby95 7y ago
The problem is that if you have one big dumb lock around everything then you have no actual concurrency or parallelism.
- 01100011 7y ago
  Depends on what the big dumb lock protects. You may still be able to process data you accessed while holding the big dumb lock.
unlinked_dll 7y ago
One big dumb lock only protects against data races. There are other classes of concurrency bugs that require smart locking to avoid, and it's always a pain. For example, iterator invalidation. Which is probably the most common thing I've seen when people use dumb locks to make a concurrent program sound. Either your iterators need to be aware of the lock and hold it through their lifetime (which means practically only one iterator can exist, which is undesirable) or your datastructure needs to be aware of outstanding iterators to it and update them (which means you have added logic to simple mutations, which is also undesirable). It's problems like that you look for in non-functional languages that require library authors to say things like "this is/isn't thread safe" in their documentation, while providing those guarantees manually. And as a consumer of the library you need to be aware of these things and look for them, and what an error looks like when someone messes up. I don't know what you're trying to say about stacks/the call stack in functional languages, because they implement their patterns in variety of ways - including locking. It's just about providing particular guarantees in the API, and the benefits of those guarantees are that it prevents entire classes of bugs like iterator invalidation and data races. There are also all sorts of clever data structures you can use that don't require duplication of the underlying data. The easiest is a singly linked list without insertion. No locks required.

Heliosmaster 7y ago

Yes. I can say that doing concurrency in Clojure is particularly easy, check for yourself: https://www.braveclojure.com/core-async/ https://www.braveclojure.com/core-async/

imtringued 7y ago

You are confusing Moores' Law with Dennard scaling. Without Moores' Law you're not adding more transistors and therefore more cores.

jimbokun 7y ago

> Is concurrency really easier in functional languages You can use the same approaches in non-functional languages. You can only use immutable data structures, and pure functions everywhere. Then you get the same benefits of easier reasoning about concurrent execution, as you get from a functional programming language. So the point of using an actual functional language, is that the compiler pushes you very strongly towards programming this way. It's kind of like using a language with built in garbage collection. You could implement a garbage collector in C, and use it everywhere in your program where you allocate and release memory. But having it build into the language frees you up from having to think about it very much.

dirtydroog 7y ago
I don't get the immutable data structures thing though. If I have a vector of 1 billion numbers and I need to sort it it'll be unfeasible to make a copy every time I swap elements. RAM isn't infinite either. So, deep down in the guts of the language's libraries there must be mutable operations. Which kind of defeats the whole point. I don't get point of functional programming. Machines don't work that way.
- tmountain 7y ago
  The implementations of immutable data structures in most concurrency aware functional languages are smarter than that. Take Clojure as an example: "All of the Clojure collections are immutable and persistent. In particular, the Clojure collections support efficient creation of 'modified' versions, by utilizing structural sharing, and make all of their performance bound guarantees for persistent use." https://hypirion.com/musings/understanding-persistent-vector-pt-1 https://hypirion.com/musings/understanding-persistent-vector...
- moring 7y ago
  IMHO the advantages of functional programming is often looked for in examples that are too small for them to exist. There is no advantage in sorting a billion numbers the functional way. There is an advantage to gain, though, when the unsorted numbers are used for other purposes besides sorting. Then, functional programming demands that the output of sorting is written to a new 1-billion array instead of "in place", so other clients can consume the same array in parallel without being disturbed by the sorting. > So, deep down in the guts of the language's libraries there must be mutable operations. Which kind of defeats the whole point. It doesn't defeat the point if the abstraction doesn't leak. If a pure functional language offers sorting as a pure primitive, then language-wise it doesn't matter that it actually mutates memory internally. The problem with "mostly" pure functional languages is that the abstraction leaks enough to make it useless.

toast0 7y ago

> Could someone with more experience comment on that? Is concurrency really easier in functional languages and is the rising importance of concurrency a valid reason to look into functional programming? Only you know if concurrency is relavant to your work. A lot of internet stuff is deeply concurrent, but there's a lot of sequential work out there, too. I have experience with Erlang. It's amazing for concurrency, but the reason is really not anything it gives you, but everything it prevents you from doing, or requires you to do. Erlang's immutability is important because it means you can't save a reference to something and have it magically updated; if you want a value updated somewhere, you need to send the current value or request the current value at time of use; explicit synchronization makes concurrency clearer, and helps influence system design to reduce synchronization. Immutability also makes message passing easier; the content of a message can't change after it's composed which means the implementation is free to make a copy of the message if it's convenient and as a programmer, there's no need to consider the effects of changing something that you sent to another process. Immutability also makes garbage collection very simple; it's impossible to form a looped datastructure, so there's no need for mark and sweep, a simple copying collector is sufficient. Individual stacks per Erlang process (like a green thread) means GC pauses don't stop the world, only individual processes, and pause length is bounded and proportional to the memory use of the process. You could certainly set up a similar environment in C or whatever traditional language; the problem would be enforcing your system design on your own code, and partitioning off library code (including libc and/or kernel syscalls) to ensure it doesn't impact your system design either.