5 ms·

the biggest gotcha I've seen with C# multithreaded is the usage of the basic .NET Collections: Dictionary<K,V> and List<V> they will mostly work (like multipl

the biggest gotcha I've seen with C# multithreaded is the usage of the basic .NET Collections: Dictionary<K,V> and List<V> they will mostly work (like multiple threads reading, or 1 writes while another retrieves), except when two threads try to simultaneously write, or when 1 writes while another enumerates. When that happens, all bets are off, and you will silently get obtuse errors such as null return values or enumerating past the end of the collection, or a corrupt collection that throws exceptions from other (seemingly) random and unrelated areas. I believe the new concurrent collections take care of this, but it is still so easy for beginners to shoot themselves with async programming. very much in stark contrast to how dev-friendly the rest of the CLR is.

dahauns 10y ago
New concurrent collections? .NET 4 is over eight years old now. And for all these years, the docs have explicitly informed about lack of thread safety in the non-concurrent versions and referenced the concurrent equivalents.
- novaleaf 10y ago
  yes, but it doens't stop novices from doing it, or perhaps if you have existing code you are trying to parallelize.
lmm 10y ago
That's pretty unfortunate by comparison with Java, where while ConcurrentModificationException is specified as unreliable you tend to hit it pretty quickly in practice (and then hopefully the docs encourage you to fix your code properly rather than catching it).
- alkonaut 10y ago
  You get that in C# as well, if you modify a collection that is being enumerated. That is simply thrown by e.g the List enumerator MoveNext() method if it discovers that the underlying list has been modified, which it detects since all mutating methods increments a version. You don't even need multiple threads for this to occur! You can just create a list enumerator, and then accidentally add/remove an item while iterating. The more difficult problems to detect are those unrelated to collection versions + enumerators. For example accidentally reading a Dictionary<K, V> while someone is adding to it on another thread (this requires true concurrency). This one has no builtin detection for modification. Does the Java dictionary really check thread identifiers for who is modifying? Or use some kind of entry counter to detect concurrency? I'd love to have that in debug builds in C#. The result in C# is usually that it works 99% of the time and corrupts silently in the edge case that someone happened to read the hash table at the same time as an internal array was resized or similar. Typical consequence is that it reports a count of N and Contains(x) is true, but when listing the contents we see some other number of items, and no sign of x... It's that customer report with a stack trace you say "that's impossible, foo is never null there". Or that unit test that only fails one build of 1000 and you blame cosmic rays first...
  - Matthias247 10y ago
    Checking Thread IDs alone won't help. It is perfectly valid that a collection or even if an iterator is passed between multiple threads and used by them, as long as there's external synchronization (which the collection implementation unfortunately can't see). So it's hard to check for the error at runtime - the only thing that can be done is invalidating old iterators if a modification happens, which can be detected later on (ConcurrentModificatoinException). As you said that will not always reliably detect the error and show it the developer. But that's imho the biggest challenge in concurrent and multithreaded programming in general: Errors are mostly not visible - until they totally tear down the software in all imaginable ways. And the root cause if often very very hard to find. There's also no general solution for improving the situation. E.g. making all collections concurrent collections would mean they would not throw exceptions anymore if misused, but most likely the application behavior would be silently broken - and the user would take a performance hit. Imho the best solution that's currently in use is preventing shared mutable objects between threads at all - like Erlang (and also Javascript!) are doing. But for general purpose languages like C# and Java people would complain that this is too opinionated and disallows several optimizations.
    - alkonaut 10y ago
      Exactly - the problem I'm usually facing with concurrent collection access in C# and Java is that the concurrent ccess wasn't even known or even intended. Having a mode or a special set of collections that guarantee that they are never accessed from two threads would be very useful to detect unintended sharing. It wouldn't help in scenarios where things are passed between threads sequentially but for that case at least the sharing is intended and can be reasoned about. The bugs are invariably in code no one even considered involved two threads at all. Having e.g a debug-only optional mode that verifies all calls to collection classes are from the same thread would provide a way of trapping many of these. C# makes it very easy to avoid the normal threading issues by using Task<T> and other high level helpers - which is excellent. The problem I'm usually facing is isolating what the code within the task can "reach". Normally the main program code is a huge amount of code that must be single threaded. Tons of caches using vanilla dictionaries and so on. Now there is a small perf critical thing I need to do with Tasks. The failure mode is that the code from in the task somehow indirectly (e.g through some property on an object ending up in a cache backed by a dictionary) ends up accessing shared mutable state. Soparating the code used concurrently from the rest becomes a mental overhead. So anything that helps me do that would help.

dahauns 10y ago

New concurrent collections? .NET 4 is over eight years old now. And for all these years, the docs have explicitly informed about lack of thread safety in the non-concurrent versions and referenced the concurrent equivalents.

novaleaf 10y ago
yes, but it doens't stop novices from doing it, or perhaps if you have existing code you are trying to parallelize.

lmm 10y ago

That's pretty unfortunate by comparison with Java, where while ConcurrentModificationException is specified as unreliable you tend to hit it pretty quickly in practice (and then hopefully the docs encourage you to fix your code properly rather than catching it).

alkonaut 10y ago
You get that in C# as well, if you modify a collection that is being enumerated. That is simply thrown by e.g the List enumerator MoveNext() method if it discovers that the underlying list has been modified, which it detects since all mutating methods increments a version. You don't even need multiple threads for this to occur! You can just create a list enumerator, and then accidentally add/remove an item while iterating. The more difficult problems to detect are those unrelated to collection versions + enumerators. For example accidentally reading a Dictionary<K, V> while someone is adding to it on another thread (this requires true concurrency). This one has no builtin detection for modification. Does the Java dictionary really check thread identifiers for who is modifying? Or use some kind of entry counter to detect concurrency? I'd love to have that in debug builds in C#. The result in C# is usually that it works 99% of the time and corrupts silently in the edge case that someone happened to read the hash table at the same time as an internal array was resized or similar. Typical consequence is that it reports a count of N and Contains(x) is true, but when listing the contents we see some other number of items, and no sign of x... It's that customer report with a stack trace you say "that's impossible, foo is never null there". Or that unit test that only fails one build of 1000 and you blame cosmic rays first...
- Matthias247 10y ago
  Checking Thread IDs alone won't help. It is perfectly valid that a collection or even if an iterator is passed between multiple threads and used by them, as long as there's external synchronization (which the collection implementation unfortunately can't see). So it's hard to check for the error at runtime - the only thing that can be done is invalidating old iterators if a modification happens, which can be detected later on (ConcurrentModificatoinException). As you said that will not always reliably detect the error and show it the developer. But that's imho the biggest challenge in concurrent and multithreaded programming in general: Errors are mostly not visible - until they totally tear down the software in all imaginable ways. And the root cause if often very very hard to find. There's also no general solution for improving the situation. E.g. making all collections concurrent collections would mean they would not throw exceptions anymore if misused, but most likely the application behavior would be silently broken - and the user would take a performance hit. Imho the best solution that's currently in use is preventing shared mutable objects between threads at all - like Erlang (and also Javascript!) are doing. But for general purpose languages like C# and Java people would complain that this is too opinionated and disallows several optimizations.
  - alkonaut 10y ago
    Exactly - the problem I'm usually facing with concurrent collection access in C# and Java is that the concurrent ccess wasn't even known or even intended. Having a mode or a special set of collections that guarantee that they are never accessed from two threads would be very useful to detect unintended sharing. It wouldn't help in scenarios where things are passed between threads sequentially but for that case at least the sharing is intended and can be reasoned about. The bugs are invariably in code no one even considered involved two threads at all. Having e.g a debug-only optional mode that verifies all calls to collection classes are from the same thread would provide a way of trapping many of these. C# makes it very easy to avoid the normal threading issues by using Task<T> and other high level helpers - which is excellent. The problem I'm usually facing is isolating what the code within the task can "reach". Normally the main program code is a huge amount of code that must be single threaded. Tons of caches using vanilla dictionaries and so on. Now there is a small perf critical thing I need to do with Tasks. The failure mode is that the code from in the task somehow indirectly (e.g through some property on an object ending up in a cache backed by a dictionary) ends up accessing shared mutable state. Soparating the code used concurrently from the rest becomes a mental overhead. So anything that helps me do that would help.