5 ms·

Clojure had STM from the start, and it was one of its selling points, but the real world experience with it is that nobody uses it - atoms (i.e. global seriali

by kimi 6y ago

Clojure had STM from the start, and it was one of its selling points, but the real world experience with it is that nobody uses it - atoms (i.e. global serialized write access, with non-locked reading) are 99.9% good enough.

_pmf_ 6y ago
It was supposed to be THE killer feature of Clojure. Rich Hickey's "ants" demo is still firmly in my mind.
MrBuddyCasino 6y ago
Also it isn’t particularly fast afaik. Hardware support was supposed to fix this, but the Intel primitives were buggy and not available on all models.
- reitzensteinm 6y ago
  Hardware transactional memory wouldn't accelerate Clojure. The actual transactional bit is very cheap, built around compare and swap. You first grab the current value of two refs, build two new values out of them through some process. Then on commit, you lock both refs and check they're unchanged since the start of the transaction. If they are, you swap the values, if they're not, you grab the new values and rerun the computation (on and on until timeout). The overhead comes from the lack of granularity - if two transactions change different keys in a map they'll conflict even though in theory they could run simultaneously (and indeed can with a hash table and HTM like TSX). And also because of the speed of persistent data structures. They're not exactly slow given the flexibility they give you, but each write involves allocation.
  - MrBuddyCasino 6y ago
    > The overhead comes from the lack of granularity - if two transactions change different keys in a map they'll conflict even though in theory they could run simultaneously (and indeed can with a hash table and HTM like TSX). Can you expand on that? Couldn't Clojure introduce a STM-aware HashMap, why would TSX help?
    - reitzensteinm 6y ago
      It could, but it just wouldn't be Clojure any more. The whole thing is built on persistent data structures, where if you modify an object you get the new version, and keep the old version unless it's garbage collected away. Further, there are guarantees that both versions perform well. This isn't just used for Clojure's STM (although it makes the STM model trivial). It's essentially the fundamental premise of the language. Functional programming with structural sharing to allow immutable data structures to be fast enough to be useful in the real world. Introducing a new data structure that was (I assume you mean) HTM aware would not offer the same guarantees. You'd be able to have two threads mutating it, but you couldn't (for instance) keep a history log of values by hanging on to a pointer to each one, that is still reasonably space efficient. TSX or other HTM would allow you to build a hash map that two threads can modify simultaneously by literally just reading and writing to it as normal (with some nasty edge cases, but fundamentally it's similar to a single thread version). The hardware keeps track of which cache lines are involved in the transaction, ensures they're not written to memory, and if there's a conflict it throws them away to retry the transactions. However in Clojure, you're not modifying the map, you're creating a new version that shares most of the structure with the old version except for your differences. If two threads modify the original hash map A, to yield B and C, all you've got now are three different trees. You might be able to invent a data structure that allow you to merge the diffs of A->B and A->C to yield D, containing both versions. But it's probably not going to be awfully fast and it's going to have corner cases that violate the transactional guarantees.
      - reitzensteinm 6y ago
        I should point out that I'm fairly familiar with the literature roughly up to the level that Clojure takes advantage of (I've used it to implement custom versions of persistent maps and RRB trees that store to disk). There may be more advances since then or techniques I'm not familiar with. I think the problem I'm describing is fundamental, but I'm not a researcher and am not claiming any advanced knowledge beyond being an experienced Clojure developer that is familiar with the internals.
- jwr 6y ago
  > Also it isn’t particularly fast afaik What exactly "isn't particularly fast"? :) In any real-world application the cost of STM machinery itself is, at a first approximation, 0. There is a cost to immutable data structures, but I don't think that's what you were talking about.
  - mrh0057 6y ago
    STM under a concurrent load can be expensive due to how modern CPUs work. Since you are updating values on different CPUs going to have cache invalidation. Then you have the issue if you are doing an expensive calculation in the transaction can cause the same transaction to fail repeatedly leading to starvation.
mullr 6y ago
It certainly seems to be rarely used. It's interesting to look at the concurrency stuff in Clojure and its uptake: - Atoms: very popular. Usually what people reach for, perhaps overly so. - STM / refs: Not commonly used, but I wonder if this is because we aren't used to really considering them as an option? - Agents: I think everybody agrees that you shouldn't use these - core.async: What many people think you should use instead - Reducers: I've never seen these used in the wild.
- minikomi 6y ago
  There's also pmap which is amazing, manifold & it's little deferred/event-ish universe.. Both have their place!
- wtetzner 6y ago
  > Atoms: very popular. Usually what people reach for, perhaps overly so. Having written Clojure professionally, and then moved on to Java (not intentionally, just kinda happened), I find myself using the same pattern in Java. AtomicReference + pcollections, along with Lombok's @Value is enough to get some pretty Clojurey concurrency patterns in Java.
jwr 6y ago
That is true if you take a narrow view/definition of "STM". It's true that atoms are enough in most cases, but the reason they work so well is because the rest of the language is designed with STM in mind. You need immutable data structures throughout the language in order to be able to use atoms for synchronization: when you access a data structure through an atom, you want to be sure you are getting your own "snapshot" that no one else will modify. This is why I frown when people say "Clojure STM is useless" or "nobody uses Clojure STM". Everybody uses the foundations, and the reason why the foundations are so useful is precisely because they were designed for STM. I've been writing Clojure apps for many years now, and it is true that atoms are usually enough. I used agents in the past for their serialization capability, but these days core.async is usually a better tool. I never found a use case for refs that could not be simplified. In most applications I wrote, transactionality was needed together with persistency, which shifted the responsibility to a database.
augustl 6y ago
Atoms - and immutable values :) Those two are really killer and solves all of my state problems, at least.
nlitened 6y ago
I remember in one of his interviews, Rich Hickey mentioned that STM isn’t in the language to be used all over the place, it’s there to make some very rarely used things possible, which otherwise would not be possible at all. Which is a very reasonable way to see it, in my opinion.
reitzensteinm 6y ago
I have used refs for a multiplayer server. Five years in to Clojure, there was finally a reason to break them out. Players were in rooms, which were updated ten times a second. All of the rooms are largely independent, meaning the best strategy to do this is to use a threadpool to update them in parallel. However, occasionally operations would require synchronization between rooms (eg, move one player from one to another). It worked pretty well.
blandflakes 6y ago
I think this mirrors how concurrency really is in many applications: * a lot of services simply defer their state management to some other service - a database or document store. Typically have to use their concurrency/transaction management * In apps that have state, a lot of times the only safety is required - this state only changes in one place, I can't afford a partial read, but otherwise I don't really care about concurrency. Atoms for this. * Sometimes there is an ordering or coordination to state, but a lot of time that is producer-consumer, so core.async can be a nice fit. * In a (relatively) small subset of cases, I actually have multiple writers whose behavior I need to coordinate. STM is perfect, and I prefer it to managing my own locks in those cases.

by kimi 6y ago

_pmf_ 6y ago

It was supposed to be THE killer feature of Clojure. Rich Hickey's "ants" demo is still firmly in my mind.

MrBuddyCasino 6y ago

Also it isn’t particularly fast afaik. Hardware support was supposed to fix this, but the Intel primitives were buggy and not available on all models.

reitzensteinm 6y ago
Hardware transactional memory wouldn't accelerate Clojure. The actual transactional bit is very cheap, built around compare and swap. You first grab the current value of two refs, build two new values out of them through some process. Then on commit, you lock both refs and check they're unchanged since the start of the transaction. If they are, you swap the values, if they're not, you grab the new values and rerun the computation (on and on until timeout). The overhead comes from the lack of granularity - if two transactions change different keys in a map they'll conflict even though in theory they could run simultaneously (and indeed can with a hash table and HTM like TSX). And also because of the speed of persistent data structures. They're not exactly slow given the flexibility they give you, but each write involves allocation.
- MrBuddyCasino 6y ago
  > The overhead comes from the lack of granularity - if two transactions change different keys in a map they'll conflict even though in theory they could run simultaneously (and indeed can with a hash table and HTM like TSX). Can you expand on that? Couldn't Clojure introduce a STM-aware HashMap, why would TSX help?
  - reitzensteinm 6y ago
    It could, but it just wouldn't be Clojure any more. The whole thing is built on persistent data structures, where if you modify an object you get the new version, and keep the old version unless it's garbage collected away. Further, there are guarantees that both versions perform well. This isn't just used for Clojure's STM (although it makes the STM model trivial). It's essentially the fundamental premise of the language. Functional programming with structural sharing to allow immutable data structures to be fast enough to be useful in the real world. Introducing a new data structure that was (I assume you mean) HTM aware would not offer the same guarantees. You'd be able to have two threads mutating it, but you couldn't (for instance) keep a history log of values by hanging on to a pointer to each one, that is still reasonably space efficient. TSX or other HTM would allow you to build a hash map that two threads can modify simultaneously by literally just reading and writing to it as normal (with some nasty edge cases, but fundamentally it's similar to a single thread version). The hardware keeps track of which cache lines are involved in the transaction, ensures they're not written to memory, and if there's a conflict it throws them away to retry the transactions. However in Clojure, you're not modifying the map, you're creating a new version that shares most of the structure with the old version except for your differences. If two threads modify the original hash map A, to yield B and C, all you've got now are three different trees. You might be able to invent a data structure that allow you to merge the diffs of A->B and A->C to yield D, containing both versions. But it's probably not going to be awfully fast and it's going to have corner cases that violate the transactional guarantees.
    - reitzensteinm 6y ago
      I should point out that I'm fairly familiar with the literature roughly up to the level that Clojure takes advantage of (I've used it to implement custom versions of persistent maps and RRB trees that store to disk). There may be more advances since then or techniques I'm not familiar with. I think the problem I'm describing is fundamental, but I'm not a researcher and am not claiming any advanced knowledge beyond being an experienced Clojure developer that is familiar with the internals.
jwr 6y ago
> Also it isn’t particularly fast afaik What exactly "isn't particularly fast"? :) In any real-world application the cost of STM machinery itself is, at a first approximation, 0. There is a cost to immutable data structures, but I don't think that's what you were talking about.
- mrh0057 6y ago
  STM under a concurrent load can be expensive due to how modern CPUs work. Since you are updating values on different CPUs going to have cache invalidation. Then you have the issue if you are doing an expensive calculation in the transaction can cause the same transaction to fail repeatedly leading to starvation.

mullr 6y ago

It certainly seems to be rarely used. It's interesting to look at the concurrency stuff in Clojure and its uptake: - Atoms: very popular. Usually what people reach for, perhaps overly so. - STM / refs: Not commonly used, but I wonder if this is because we aren't used to really considering them as an option? - Agents: I think everybody agrees that you shouldn't use these - core.async: What many people think you should use instead - Reducers: I've never seen these used in the wild.

minikomi 6y ago
There's also pmap which is amazing, manifold & it's little deferred/event-ish universe.. Both have their place!
wtetzner 6y ago
> Atoms: very popular. Usually what people reach for, perhaps overly so. Having written Clojure professionally, and then moved on to Java (not intentionally, just kinda happened), I find myself using the same pattern in Java. AtomicReference + pcollections, along with Lombok's @Value is enough to get some pretty Clojurey concurrency patterns in Java.

jwr 6y ago

That is true if you take a narrow view/definition of "STM". It's true that atoms are enough in most cases, but the reason they work so well is because the rest of the language is designed with STM in mind. You need immutable data structures throughout the language in order to be able to use atoms for synchronization: when you access a data structure through an atom, you want to be sure you are getting your own "snapshot" that no one else will modify. This is why I frown when people say "Clojure STM is useless" or "nobody uses Clojure STM". Everybody uses the foundations, and the reason why the foundations are so useful is precisely because they were designed for STM. I've been writing Clojure apps for many years now, and it is true that atoms are usually enough. I used agents in the past for their serialization capability, but these days core.async is usually a better tool. I never found a use case for refs that could not be simplified. In most applications I wrote, transactionality was needed together with persistency, which shifted the responsibility to a database.

augustl 6y ago

Atoms - and immutable values :) Those two are really killer and solves all of my state problems, at least.

nlitened 6y ago

I remember in one of his interviews, Rich Hickey mentioned that STM isn’t in the language to be used all over the place, it’s there to make some very rarely used things possible, which otherwise would not be possible at all. Which is a very reasonable way to see it, in my opinion.

reitzensteinm 6y ago

I have used refs for a multiplayer server. Five years in to Clojure, there was finally a reason to break them out. Players were in rooms, which were updated ten times a second. All of the rooms are largely independent, meaning the best strategy to do this is to use a threadpool to update them in parallel. However, occasionally operations would require synchronization between rooms (eg, move one player from one to another). It worked pretty well.

blandflakes 6y ago

I think this mirrors how concurrency really is in many applications: * a lot of services simply defer their state management to some other service - a database or document store. Typically have to use their concurrency/transaction management * In apps that have state, a lot of times the only safety is required - this state only changes in one place, I can't afford a partial read, but otherwise I don't really care about concurrency. Atoms for this. * Sometimes there is an ordering or coordination to state, but a lot of time that is producer-consumer, so core.async can be a nice fit. * In a (relatively) small subset of cases, I actually have multiple writers whose behavior I need to coordinate. STM is perfect, and I prefer it to managing my own locks in those cases.