22 ms·

The two sentences are consistent once you read that Go maps are not designed to be safe for concurrent access.

by iand 11y ago

grabcocque 11y ago
Java's (pre-concurrent) HashMaps aren't designed for concurrent access. Do you know what they do? They throw a ConcurrentModificationException. Do you know what they don't do? Silently corrupt themselves.
- vbezhenar 11y ago
  From HashMap JavaDoc: Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs. So even while Java tries to find concurrency bugs, it does not promise it.
  - 4ad 11y ago
    Not to mention that Go does have the race detector, which does the same thing in this case (although it does so much more in general).
  - lmm 11y ago
    Indeed it doesn't promise it. But in practice it's very good at it - the reason they have to put that note in there is because if you made a program whose correctness depended on it, it would be a long time before you saw the bug.
- alkonaut 11y ago
  That check probably comes at a performance cost that the Go standard library designers found unacceptable. You don't even need to be modifying it concurrently to read inconsistent state, it's enough to write on one thread and read on another, while the map is updating internally. So in order to prevent inconsistent reads you would have to check for concurrent access on all reads. The .NET BCL Dictionary is just like the Go one: it does no checking for you and will silently corrupt or return an inconsistent view if you access it concurrently. The whole reason there are two implementations of the dictionary (Concurrent and Non concurrent) is that the performance cost of the concurrent implementation is too high to carry for the non-concurrent one. Depending on the cost of checking for concurrent access, a lot of the gain of using a simpler non-concurrent one could be lost.
  - shellac 11y ago
    The check is actually very cheap -- you can check the source yourself -- but as you might expect it won't catch all unsafe use.
    - alkonaut 11y ago
      Like you suggest you can make a cheap check that sometimes triggers an exception. Or an expensive check that always triggers on all concurrent access. As far as I can see in the java HashMap source, the concurrent modification check is actually only done for the enumerator (?) so the problem of multiple concurrent insertions, or even reading-while-inserting will never be caught anyway. Just like Go, or .NET. A check for single thread access (which is even stricter than non-concurrent access which allows several threads as long as they aren't used concurrently) would be pretty cheap: store the thread ID on creation and then verify on reads and writes. Failing on concurrent access otherwise on all reads and writes would basically mean that you add a lock to the write operation, and fail any reads while anyone is writing. This however is close enough to making it a full concurrent map, so it isn't worth it.
  - binarycrusader 11y ago
    Well, they believed it was worth adding something (from a commit today): https://github.com/golang/go/commit/50c5042047be3af36e7bb478435093ea45e8f1f0 https://github.com/golang/go/commit/50c5042047be3af36e7bb478...
    - alkonaut 11y ago
      That's interesting. Would have liked seeing something like that in the Java and .NET equivalents.
- noselasd 11y ago
  A Java Hastable is threadsafe, and is documented to be so A Java HashMap is not threadsafe and is documented to be so, and can silently corrupt data, can go into an infinite loop etc. if you mess with it from several threads. Both leaves you with a lot of potential for races in application logic though.
coldtea 11y ago
Yeah, but the parent's phrase was: "If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong". If we accept his premise, then the fact that "they wasn't designed for that" is no excuse. It's like selling a kids toy that has tiny choke-inducing parts. In that case, just saying: "It wasn't designed to be eaten by kids" is not really an excuse.
- arethuza 11y ago
  I think there is a big difference between "being good for concurrency" and preventing all concurrent programming problems - concurrency is hard and while go does seem to provide a lot of nice structures to make it more straightforward it can't, and doesn't appear to claim to, hide all the complexities of concurrency from developers.
- 4ad 11y ago
  First, that statement was not present in grandparent's post when the parent posted his message. Second, that premise is completely untrue. Go, and most other languages give you more-or-less orthogonal primitives and let you combine them. It's okay for your types not to be thread safe, if by combining them with some other primitive you can make them thread safe. Every other language does this. Some offer you a concurrent map along with a non-concurrent map. For many reasons I won't get into, Go only has one kind of map and lets the user do the rest (which is an extremely easy idiom in Go). Note that even though many languages conveniently offer concurrent maps, no languages with mutable state that I know of offer concurrent integer and concurrent structs. The user is still responsible for ensuring the safety of those. This is perfectly fine because the grandparent's premise is wrong. In concurrent language, by far the most common case is by data to be owned by a goroutine/thread/whatever. It's very easy to reason about code this way, and it's the way you are encouraged to write code. Only when you need to share data you need to worry about concurrency-safety, and in that scenario you have available all the tools to ensure it.
  - lambda_cube 11y ago
    > no languages with mutable state that I know of offer concurrent integer and concurrent structs Java does for integer http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicInteger.html http://docs.oracle.com/javase/8/docs/api/java/util/concurren... You can find similar classes in the java.util.concurrent.atomic package. http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/package-summary.html http://docs.oracle.com/javase/8/docs/api/java/util/concurren... I'm not sure what requirements you have for a concurrent struct, but Java has classes to atomically manipulate int and long fields of a class. I'm not arguing your general point (in fact, I agree with it), I'm just supplying some extra information of a language that you apparently don't know.

The two sentences are consistent once you read that Go maps are not designed to be safe for concurrent access.

by iand 11y ago

The two sentences are consistent once you read that Go maps are not designed to be safe for concurrent access.

Java's (pre-concurrent) HashMaps aren't designed for concurrent access. Do you know what they do? They throw a ConcurrentModificationException. Do you know what they don't do? Silently corrupt themselves.

vbezhenar 11y ago
From HashMap JavaDoc: Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs. So even while Java tries to find concurrency bugs, it does not promise it.
- 4ad 11y ago
  Not to mention that Go does have the race detector, which does the same thing in this case (although it does so much more in general).
- lmm 11y ago
  Indeed it doesn't promise it. But in practice it's very good at it - the reason they have to put that note in there is because if you made a program whose correctness depended on it, it would be a long time before you saw the bug.
alkonaut 11y ago
That check probably comes at a performance cost that the Go standard library designers found unacceptable. You don't even need to be modifying it concurrently to read inconsistent state, it's enough to write on one thread and read on another, while the map is updating internally. So in order to prevent inconsistent reads you would have to check for concurrent access on all reads. The .NET BCL Dictionary is just like the Go one: it does no checking for you and will silently corrupt or return an inconsistent view if you access it concurrently. The whole reason there are two implementations of the dictionary (Concurrent and Non concurrent) is that the performance cost of the concurrent implementation is too high to carry for the non-concurrent one. Depending on the cost of checking for concurrent access, a lot of the gain of using a simpler non-concurrent one could be lost.
- shellac 11y ago
  The check is actually very cheap -- you can check the source yourself -- but as you might expect it won't catch all unsafe use.
  - alkonaut 11y ago
    Like you suggest you can make a cheap check that sometimes triggers an exception. Or an expensive check that always triggers on all concurrent access. As far as I can see in the java HashMap source, the concurrent modification check is actually only done for the enumerator (?) so the problem of multiple concurrent insertions, or even reading-while-inserting will never be caught anyway. Just like Go, or .NET. A check for single thread access (which is even stricter than non-concurrent access which allows several threads as long as they aren't used concurrently) would be pretty cheap: store the thread ID on creation and then verify on reads and writes. Failing on concurrent access otherwise on all reads and writes would basically mean that you add a lock to the write operation, and fail any reads while anyone is writing. This however is close enough to making it a full concurrent map, so it isn't worth it.
- binarycrusader 11y ago
  Well, they believed it was worth adding something (from a commit today): https://github.com/golang/go/commit/50c5042047be3af36e7bb478435093ea45e8f1f0 https://github.com/golang/go/commit/50c5042047be3af36e7bb478...
  - alkonaut 11y ago
    That's interesting. Would have liked seeing something like that in the Java and .NET equivalents.
noselasd 11y ago
A Java Hastable is threadsafe, and is documented to be so A Java HashMap is not threadsafe and is documented to be so, and can silently corrupt data, can go into an infinite loop etc. if you mess with it from several threads. Both leaves you with a lot of potential for races in application logic though.

coldtea 11y ago

Yeah, but the parent's phrase was: "If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong". If we accept his premise, then the fact that "they wasn't designed for that" is no excuse. It's like selling a kids toy that has tiny choke-inducing parts. In that case, just saying: "It wasn't designed to be eaten by kids" is not really an excuse.

arethuza 11y ago
I think there is a big difference between "being good for concurrency" and preventing all concurrent programming problems - concurrency is hard and while go does seem to provide a lot of nice structures to make it more straightforward it can't, and doesn't appear to claim to, hide all the complexities of concurrency from developers.
4ad 11y ago
First, that statement was not present in grandparent's post when the parent posted his message. Second, that premise is completely untrue. Go, and most other languages give you more-or-less orthogonal primitives and let you combine them. It's okay for your types not to be thread safe, if by combining them with some other primitive you can make them thread safe. Every other language does this. Some offer you a concurrent map along with a non-concurrent map. For many reasons I won't get into, Go only has one kind of map and lets the user do the rest (which is an extremely easy idiom in Go). Note that even though many languages conveniently offer concurrent maps, no languages with mutable state that I know of offer concurrent integer and concurrent structs. The user is still responsible for ensuring the safety of those. This is perfectly fine because the grandparent's premise is wrong. In concurrent language, by far the most common case is by data to be owned by a goroutine/thread/whatever. It's very easy to reason about code this way, and it's the way you are encouraged to write code. Only when you need to share data you need to worry about concurrency-safety, and in that scenario you have available all the tools to ensure it.
- lambda_cube 11y ago
  > no languages with mutable state that I know of offer concurrent integer and concurrent structs Java does for integer http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicInteger.html http://docs.oracle.com/javase/8/docs/api/java/util/concurren... You can find similar classes in the java.util.concurrent.atomic package. http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/package-summary.html http://docs.oracle.com/javase/8/docs/api/java/util/concurren... I'm not sure what requirements you have for a concurrent struct, but Java has classes to atomically manipulate int and long fields of a class. I'm not arguing your general point (in fact, I agree with it), I'm just supplying some extra information of a language that you apparently don't know.