4 ms·

I thought it was a bit odd that the author claims there’s no mutexes in sight, the TVar is effectively a mutex guard unless I’m misunderstanding this? (I’ve wri

I thought it was a bit odd that the author claims there’s no mutexes in sight, the TVar is effectively a mutex guard unless I’m misunderstanding this? (I’ve written exactly 0 lines of Haskel). Or is the claim that the lack of ceremony and accidental complexity around threading is the real win for concurrency here?

chongli 1y ago
No, a TVar is not a mutex guard. A TVar is a software transactional memory (STM) variable. STM works just like a database: you batch together a sequence of operations into a transaction and then execute them. During execution of a transaction, all changes made to the contents of the TVar are stored in a transaction log. If some other transaction occurs during the execution then the whole thing is aborted and re-run. This can take any ordinary Haskell data structure and give you a lock-free concurrent data structure with easy-to-use transactional semantics. How it performs is another matter! That depends on the amount of contention and the cost of re-playing transactions.
- whateveracct 1y ago
  https://hackage.haskell.org/package/stm-containers https://hackage.haskell.org/package/stm-containers This library is full of STM-oriented data structures. They perform better than a simple `TVar (Map k v)`. It's kind of a fun trick actually. The stock Map is just a tree. The STM Map is also a tree [1] but with TVars at each node. So this helps a lot with contention - you only contend along a "spine" instead of across the whole tree, which is O(log n). [1] Technically a HAMT a la unordered-containers - trie, tree, you get the idea :)
- quibono 1y ago
  > How it performs is another matter! I know you say it depends on how much contention one sees but I'm interested in the performance hit. Also, is STM the "standard" (or accepted) way to do async in Haskell?
pkjens04 1y ago
A mutex locks the thread, which doesn't happen with TVar. For mutexes TMVar would be used.
dsign 1y ago
You are correct, Haskell has quite a few mutex-like types. MVar is one of them. However, if memory serves me right, TVar is a building block for the transactional memory subsystem. The guard on TVar with, say, modifyTVar is not really stopping execution at entrance but simply indicating that the block modifies the variable. In my mental model, some magic happens in an STM block that checks if two concurrent STM blocks acted upon the same data at the same time, and if so, it reverts the computations of one of the blocks and repeats them with new data. To my knowledge, Haskell is the only programming language (+runtime) that has a working transactional memory subsystem. It has been in the language for about 20 years, and in that time many have tried (and failed) to also implement STM.
- dionian 1y ago
  https://zio.dev/reference/stm/ https://zio.dev/reference/stm/
  - mrkeen 1y ago
    > Implication of Using STM Running I/O Inside STM— There is a strict boundary between the STM world and the ZIO world. This boundary propagates even deeper because we are not allowed to execute arbitrary effects in the STM universe. Performing side effects and I/O operations inside a transaction is problematic. In the STM the only effect that exists is the STM itself. We cannot print something or launch a missile inside a transaction as it will nondeterministically get printed on every reties that transaction does that. Does Zio actually offer any protection here, or is it just telling the reader that they're on their own and should be wary of footguns?
    - dionian 1y ago
      I am not super familiar with it, great question - my guess would be its the latter! Does Haskell's provide protections?
    - hackingonempty 1y ago
      STM happens inside the STM monad while regular effects happen in the ZIO monad. If you try to do ZIO effects inside an STM transaction you'll get a type error. Scala doesn't enforce purity like Haskell though so it wont stop you if you call some normal Scala or Java code with side effects. In practice its not a problem because you're wrapping any effectful outside APIs before introducing them into your code.
dwohnitmok 1y ago
No a TVar isn't a mutex guard. As a sibling comment points out it gives you transactional semantics similar to most relational databases. Here's an example in perhaps more familiar pseudocode. var x = "y is greater than 0" var y = 1 forkAndRun {() => y = y - 1 if (y <= 0) { x = "y is less than or equal to 0" } } forkAndRun {() => y = y + 1 if (y > 0) { x = "y is greater than 0" } } In the above example, it's perfectly possible, depending on how the forked code blocks interact with each other, to end up with x = "y is less than or equal to 0" y = 1 because we have no guarantee of atomicity/transactionality in what runs within the `forkAndRun` blocks. The equivalent of what that Haskell code is doing is replacing `var` with a new keyword `transactional_var` and introducing another keyword `atomically` such that we can do transactional_var x = "y is greater than 0" transactional_var y = 1 forkAndRun { atomically {() => y = y - 1 if (y <= 0) { x = "y is less than or equal to 0" } } } forkAndRun { atomically {() => y = y + 1 if (y > 0) { x = "y is greater than 0" } } } and never end up with a scenario where `x` and `y` disagree with each other, because all their actions are done atomically together and `x` and `y` are specifically marked so that in an atomic block all changes to the variables either happen together or are all rolled back together (and tried again), just like in a database. `transactional_var` is the equivalent of a `TVar` and `atomically` is just `atommically`.
mrkeen 1y ago
Mutexes lock code, TVars lock data. If you lock a section of code (to protect data), there's no guarantee against mutations of that data from other sections of code. If you lock the data itself, you can freely pass it around and anyone can operate on it concurrently (and reason about it as if it were single-threaded). It's the same approach as a transactional database, where you share one gigantic bucket of mutable state with many callers, yet no-one has to put acquire/release/synchronise into their SQL statements.
ghusbands 1y ago
As siblings note, TVar is a transactional variable. However, it's not just protective against concurrent writes but also against concurrent reads of altered variables, so it offers true atomicity across any accessed state in a transaction. So if you have a thread altering `foo` and checking that `foo+bar` isn't greater than 5 and a thread altering `bar` and checking the same, then it's guaranteed that `foo+bar` does not exceed 5. Whereas if only write conflicts were detected (as is default with most databases) then `foo+bar` could end up greater than 5 through parallel changes.

chongli 1y ago

No, a TVar is not a mutex guard. A TVar is a software transactional memory (STM) variable. STM works just like a database: you batch together a sequence of operations into a transaction and then execute them. During execution of a transaction, all changes made to the contents of the TVar are stored in a transaction log. If some other transaction occurs during the execution then the whole thing is aborted and re-run. This can take any ordinary Haskell data structure and give you a lock-free concurrent data structure with easy-to-use transactional semantics. How it performs is another matter! That depends on the amount of contention and the cost of re-playing transactions.

whateveracct 1y ago
https://hackage.haskell.org/package/stm-containers https://hackage.haskell.org/package/stm-containers This library is full of STM-oriented data structures. They perform better than a simple `TVar (Map k v)`. It's kind of a fun trick actually. The stock Map is just a tree. The STM Map is also a tree [1] but with TVars at each node. So this helps a lot with contention - you only contend along a "spine" instead of across the whole tree, which is O(log n). [1] Technically a HAMT a la unordered-containers - trie, tree, you get the idea :)
quibono 1y ago
> How it performs is another matter! I know you say it depends on how much contention one sees but I'm interested in the performance hit. Also, is STM the "standard" (or accepted) way to do async in Haskell?

pkjens04 1y ago

A mutex locks the thread, which doesn't happen with TVar. For mutexes TMVar would be used.

dsign 1y ago

You are correct, Haskell has quite a few mutex-like types. MVar is one of them. However, if memory serves me right, TVar is a building block for the transactional memory subsystem. The guard on TVar with, say, modifyTVar is not really stopping execution at entrance but simply indicating that the block modifies the variable. In my mental model, some magic happens in an STM block that checks if two concurrent STM blocks acted upon the same data at the same time, and if so, it reverts the computations of one of the blocks and repeats them with new data. To my knowledge, Haskell is the only programming language (+runtime) that has a working transactional memory subsystem. It has been in the language for about 20 years, and in that time many have tried (and failed) to also implement STM.

dionian 1y ago
https://zio.dev/reference/stm/ https://zio.dev/reference/stm/
- mrkeen 1y ago
  > Implication of Using STM Running I/O Inside STM— There is a strict boundary between the STM world and the ZIO world. This boundary propagates even deeper because we are not allowed to execute arbitrary effects in the STM universe. Performing side effects and I/O operations inside a transaction is problematic. In the STM the only effect that exists is the STM itself. We cannot print something or launch a missile inside a transaction as it will nondeterministically get printed on every reties that transaction does that. Does Zio actually offer any protection here, or is it just telling the reader that they're on their own and should be wary of footguns?
  - dionian 1y ago
    I am not super familiar with it, great question - my guess would be its the latter! Does Haskell's provide protections?
  - hackingonempty 1y ago
    STM happens inside the STM monad while regular effects happen in the ZIO monad. If you try to do ZIO effects inside an STM transaction you'll get a type error. Scala doesn't enforce purity like Haskell though so it wont stop you if you call some normal Scala or Java code with side effects. In practice its not a problem because you're wrapping any effectful outside APIs before introducing them into your code.

dwohnitmok 1y ago

No a TVar isn't a mutex guard. As a sibling comment points out it gives you transactional semantics similar to most relational databases. Here's an example in perhaps more familiar pseudocode. var x = "y is greater than 0" var y = 1 forkAndRun {() => y = y - 1 if (y <= 0) { x = "y is less than or equal to 0" } } forkAndRun {() => y = y + 1 if (y > 0) { x = "y is greater than 0" } } In the above example, it's perfectly possible, depending on how the forked code blocks interact with each other, to end up with x = "y is less than or equal to 0" y = 1 because we have no guarantee of atomicity/transactionality in what runs within the `forkAndRun` blocks. The equivalent of what that Haskell code is doing is replacing `var` with a new keyword `transactional_var` and introducing another keyword `atomically` such that we can do transactional_var x = "y is greater than 0" transactional_var y = 1 forkAndRun { atomically {() => y = y - 1 if (y <= 0) { x = "y is less than or equal to 0" } } } forkAndRun { atomically {() => y = y + 1 if (y > 0) { x = "y is greater than 0" } } } and never end up with a scenario where `x` and `y` disagree with each other, because all their actions are done atomically together and `x` and `y` are specifically marked so that in an atomic block all changes to the variables either happen together or are all rolled back together (and tried again), just like in a database. `transactional_var` is the equivalent of a `TVar` and `atomically` is just `atommically`.

mrkeen 1y ago

Mutexes lock code, TVars lock data. If you lock a section of code (to protect data), there's no guarantee against mutations of that data from other sections of code. If you lock the data itself, you can freely pass it around and anyone can operate on it concurrently (and reason about it as if it were single-threaded). It's the same approach as a transactional database, where you share one gigantic bucket of mutable state with many callers, yet no-one has to put acquire/release/synchronise into their SQL statements.

ghusbands 1y ago

As siblings note, TVar is a transactional variable. However, it's not just protective against concurrent writes but also against concurrent reads of altered variables, so it offers true atomicity across any accessed state in a transaction. So if you have a thread altering `foo` and checking that `foo+bar` isn't greater than 5 and a thread altering `bar` and checking the same, then it's guaranteed that `foo+bar` does not exceed 5. Whereas if only write conflicts were detected (as is default with most databases) then `foo+bar` could end up greater than 5 through parallel changes.